Cloud Computing Spring 2024
Assignment 1: Building an Automatically Scaling Web Application
Deadline: Monday, April 15, 2024
1 Aim and Scope
In this assignment we will build a small automatically scaling testbed for a (very) trivial Web
application. The goal of the assignment is to become familiar with all facets of scaling Web
applications, which will increase your understanding of the low-level / fundamental implementation
details of Cloud systems. As we have discussed in class, we could deploy such a web application
within a virtual machine or within containers. To keep things manageable on a single workstation,
or laptop, we will constrain ourselves to a single host and make use of containers. In order to
simulate saturated servers, the Web application will be single-threaded and can be rate limited on
purpose. Do note that this assignment can be implemented similarly using virtual machines, and
also can be easily extended to a distributed system consisting of multiple hosts. Both of these are
out of scope of this assignment however.
As has been discussed during the lectures, a number of components are required to create an
automatically scaling Web application. These components are summarized in the following figure:
Load
generator
("client")
Load
balancer
Web
application
container
Web
application
container
Web
application
container
...
Shared
data
internal
container
network
Scaling
controller
monitors
load balancer
instructs
container engine
to start/stop
instances
will run in container engine
As can be seen, the assignment is centered around a Web application, deployed in a container,
that can be scaled out when the incoming load requires so. A load balancer is used to balance
the load over multiple instances of the Web application (or a single instance if this is sufficient to
sustain the load). It is the responsibility of the scaling controller to monitor the incoming load
and to scale out or scale in when required. The load generator is used in experiments to evaluate
the system.
1
2 Requirements
The goal of this assignment is to create the setup as depicted in the above figure and to perform
a number of experiments. In a report that must accompany your submission, your setup needs to
be well documented, the architecture must be described, and must include the experiments that
have been conducted. As an actual Web application to scale you will be provided with a small
and trivial object store API. This application is single-threaded, can be rate limited on purpose
and can insert random delays in serving HTTP requests to mimic a server under load (both of
which are configurable). See Appendix A for details.
Within this assignment, you will be using podman as container engine. As described above,
we have chosen to use containers for this assignment because it is easier to work with than a
fully fledged virtual machine hypervisor and requires (significantly) less memory and disk space
for hosting multiple instances of the developed Web application. Some tips & tricks using podman
can be found in Appendix B.
The scaling controller and algorithm are to be designed and developed by you. You have free￾dom in designing the scaling algorithm. This algorithm can be rule-based, based on regression, or
use a sliding window, etc., etc. We recommend that you conduct a number of ‘calibration’ exper￾iments in conjunction with the development of the scaling algorithm (see also in the enumeration
below). The information obtained through calibration should be used to design and tune a scaling
algorithm for your scaling controller. For instance, design rules/thresholds for scaling decisions.
It might be worthwhile to first set a particular target response time, so to make a Service Level
Agreement, that you want to achieve also under increasing client load. Finally, the project is
concluded with a set of experiments to evaluate the effectiveness of your scaling controller.
The following table summarizes the requirements regarding the different system components:
Component Requirement
Web application Provided on Brightspace.
Container engine podman must be used.
Load balancer Required. You are allowed to write your own (simple) load balancer,
but are also free to pick an existing one such as HAProxy.
Load generator Required. You are allowed to either write your own request generator
or to use an existing one (e.g., jMeter (Java based) or Locust (Python
based)). We strongly recommend you to first design your experiments
and then see whether an existing project can be used to perform your
experiments.
Scaling controller Required. You must write your own scaling controller. Informa￾tion is to be collected from the load balancer and/or Web application
instances. The scaling decision is carried out by executing podman
commands or API calls. See also Appendix D. This scaling controller
is to be implemented in a programming language of choice.
With regard to the experiments to be conducted within the project and to be presented and
discussed in the report:
Experiment Description
Functionality test Perform a functionality experiment (or integration test) for the load
generator and container engine. This is to make sure that the basics
work well.
Calibration experi￾ments
Required. Conduct a number of calibration experiments. You likely
want to conduct these experiments in conjunction with developing your
scaling algorithm.
(a) Determine the saturation point of a single container. Play with rate
limiting and the random delays to see what effects this has on response
time (latency), request throughput, CPU utilization, etc.
2
(b) A similar experiment, with different amounts of containers, to get
an impression of how the performance would scale.
(c) Determine the time required to spawn new containers.
Final experiment Required. Conduct two meaningful final experiments that evaluate
and demonstrate the effectiveness of the automatic scaling that you
have implemented. These experiments must investigate the response
of the system to an increasing and decreasing number of client requests
as to demonstrate that the system will automatically scale out and in.
Investigate the response time of the scaling system and try to improve
this to show that the number of failed requests can be minimized or
(preferably) fully eliminated such that some SLA is adhered to. Things
to experiment with include changing the rules for scaling decisions used
by the scaling controller, optimizing the method or timing of spawning
new containers (such that they can be spawned in less time), reducing
oscillation, etc.
3 Development Environment
To work on this assignment you need administrative (root) access to a Linux installation. This
could be your own laptop or workstation, however we strongly recommend to create a dedicated
virtual machine for this assignment. You can use a setup similar to your setup for Homework 1.
Make sure to assign multiple cores to this virtual machine, this is important for the experiments.
The required disk space for the virtual machine depends on the OS you want to use within the
containers. In the case of the small Alpine OS, 4 to 5 GB should be enough, otherwise consider
10 GB. Document the Linux distribution used in your report.
4 Submission and Assessment
Teams may be formed that consist of at most two persons. In a team of two members, we
expect that both members contribute to the implementation of the system and execution of the
experiments. The deadline is Monday, April 15, 2024. Submit your assignments according to the
instructions below. In case there are problems with the team work, contact the lecturer by e-mail.
As part of the report you must list the contributions of each team member to the project.
The maximum grade that can be obtained is 10. The grade is the sum of the brackets. When
scoring each component we will consider whether the functionality is complete and works, as well
as own initiatives and ideas that clearly surpass the assignment’s requirements.
❼ [4 out of 10] Completeness and functionality of the submission.
❼ [2 out of 10] Quality of the content and layout of the report.
❼ [1 out of 10] Calibration experiments (design and report).
❼ [3 out of 10] Quality and depth of the conducted experiments to evaluate the effectiveness
of the scaling controller. This comprises the experimental design, implementation of the
necessary load generator, reporting and interpretation of the results.
Assignments must be submitted through Brightspace. For each team a single submission is
expected. Please note your names and student IDs in the text box in the submission website.
Ensure that all files that are submitted include names and student IDs.
3
The following needs to be submitted:
Web Application ❼ Listing of commands or Containerfile/Dockerfile to generate the con￾tainer image for the web application.
❼ Web application source, if modifications were made.
Load balancer ❼ Configuration file.
❼ If developed by yourself: source code.
❼ If run within a container, listing of commands or Containerfile or
Dockerfile to generate this container image.
Scaling controller ❼ Source code.
Request generator ❼ If developed by yourself: source code of the generator.
❼ Configuration files, and source code of associated extension modules
if required.
Report Report in PDF format (please no Word files), in which the following is
described:
❼ Description and explanation of the implemented architecture.
❼ Include your own system diagram that reflects your designed and
implemented architecture.
❼ Description of development environment (e.g. which Linux distribu￾tion was used in your virtual machine).
❼ Choices made during implementation.
❼ A clear explanation of the scaling policy design. Use a diagram or
concise pseudocode listing in your explanation.
❼ Report on the design and results of the ‘calibration’ experiments.
❼ Report on the design and the results of the conducted experiments
to evaluate the effectiveness of your scaling controller. This includes
motivation and implementation of a load generator.
❼ List the contributions made by each team member to the assignment,
so it is clear who worked on which parts. Note that we expect that
both team members contribute to the implementation of the system
and execution of the experiments.
Finally, please note the following:
❼ All submitted source code and reports may be subject to (automatic) plagiarism checks
using Turnitin and/or MOSS. Suspicions of fraud and plagiarism will be reported to the
Board of Examiners.
❼ The use of text or code generated by ChatGPT or other AI tools is not allowed. You are
required to implement the requested source code yourself, and to write the report yourself.
❼ We may always invite teams to elaborate on their submission in an interview in case parts
of the source code or report need further explanation.
❼ As with all other course work, keep assignment solutions to yourself. Do not post the code
on public Git or code snippet repositories where it can be found by other students. If you
use Git, make sure your repository is private.
Appendices
In these appendices we have collected some background information on the Web application that
is provided to you, a number of directions and suggestions for using Podman, and some general
tips and tricks for components of the assignment.
4
A Web Application API
As Web application you are provided with a trivial object store application based on a RESTful
API. The application is written in Python using the Flask web framework. Objects are stored
by name (key or object ID) in a directory on the file system. This object ID is unique. The
application only works with text files (so not with binary image or PDF files), keep that in mind
in case you want to experiment with large files.
To deploy the application within a container, we recommend to install Python 3 packages
within the container image first. After that, create a Python venv1 and install flask limiter
and flask restful using pip.
In order to make the containers that host the Web application stateless, an external directory
will be mounted within the containers (so all containers have access to the same external directory
and thus to the same collection of objects). To keep things simple we did not consider problems
arising from concurrent access to the objects such as race conditions (we will leave this for another
course).
The application allows rate limiting and random delays to be configured. These customization
opportunities are there to support the experiments to test the scaling functionality. For the
configuration of rate limiting, refer to the bottom of the source file of the web application. By
default random delays are introduced when serving HTTP requests, to mimic a server under load.
These can be configured (and also be disabled) using the variables just above the definition of the
function random delay.
The API implemented is as follows:
GET / Return a list of object IDs.
DELETE / Delete all objects.
GET /objs/<obj id> Return content of object with ID <obj id> or 404 if object
does not exist.
PUT /objs/<obj id> Store provided content within object with ID <obj id>.
Creates a new object if an object with this ID does not yet
exist, otherwise it overwrites the existing object.
DELETE /objs/<obj id> Delete the object with specified ID. Returns 404 if the spec￾ified object does not exist.
GET /objs/<obj id>/compress Return BZ2-compressed object in base64 encoding of object
with ID <obj id> or 404 if object does not exist.
B Working with podman
We will be using podman as container engine. podman automatically handles the virtual network
and additionally provides an easy way to store state outside of the containers in order to make
our container that hosts the Web application stateless — important to simplify scaling!
To install podman refer to your solution for Homework 1, or use the development virtual
image provided by us (see Brightspace). We strongly suggest to create all containers as the root
user, to ensure that networking performs as expected. In this case, containers will be stored in
/var/lib/containers. How much storage space you need depends on what Linux distribution
you want to use within your containers. If this is a tiny distribution like Alpine, then 200 to 300
MB storage space should be enough. If you want to use a distribution like Debian, Ubuntu or
CentOS, count on 2 to 3 GB.
B.1 Commands to create container images
podman can create containers using Dockerfiles (or Containerfiles with the same syntax). Many
use another tool called buildah (which actually came before podman) to build container images.
1https://docs.python.org/3/tutorial/venv.html
5
buildah can be installed on Ubuntu/Debian with a single apt-get install buildah, or dnf
install buildah on Red Hat derivatives. buildah can handle Dockerfiles, but can also build
images from scratch. An advantage of building images from scratch is that you have full control
and can optimize for size.
A new container image can be created from scratch by starting with a base OS image. After
image creation, commands can be run inside the container to continue configuration. For the
scaling web application, you will create containers that are based on a particular container image
(see later on).
In the interest of saving disk space, one can consider to use Alpine Linux. This is a (very)
small Linux distribution that is often used within containers. To initialize a new container image
based on Alpine Linux:
container=$(buildah from alpine)
Subsequently, we can run commands within the container2
:
buildah run $container -- ls /etc
Within the container, you might want to install additional packages (Python anyone?). The exact
instructions depend on the distribution you have selected. In case you are working with Alpine,
you can install Python 3 by executing the following commands:
buildah run $container -- apk update
buildah run $container -- apk add python3
You can search the package database as follows:
buildah run $container -- apk list ’*haproxy*’
(yes, Alpine also has HAProxy packages in case you want to create a container for your load
balancer).
Files can be copied into the container using the copy subcommand, e.g.:
buildah copy $container myfile.txt /root
Finally, it is important that the changes made are committed to a named image. In this case we
will use testcontainer as name. After commit, the image will be visible within podman as well.
buildah commit $container testcontainer
Note that the same can be achieved using a Dockerfile with RUN and COPY verbs. The command
to build containers from Dockerfiles is buildah bud.
Additional Resources For more information on buildah, refer to for instance:
https://appfleet.com/blog/everything-you-need-to-know-about-buildah/
https://github.com/containers/buildah/tree/main/docs/tutorials
B.2 Starting the Web application on container startup
Do not forget to ensure that your Web application is started on container startup. This can be
achieved by configuring a container command or entrypoint. For example, to start the Python
built-in webserver, which serves files from the specified directory, upon container start up:
buildah config --cmd "" $container
buildah config --entrypoint "python3 -m http.server 8000 --directory /tmp" $container
buildah commit $container testcontainer
2On Debian 11 this does not appear to work out of the box, the following environment variable is required:
export BUILDAH RUNTIME=/usr/bin/runc.
6
B.3 Starting container instances
From images, container instances can be started. Multiple container instances can be started
from a single image, and this is exactly what we need to ‘scale out’ the Web application. This is
achieved using the podman command. Note that this will not create full copies of the container
image, rather, the image remains read-only and an overlay file system is placed on top of it to
catch any writes.
The container for your Web application is supposed to be stateless. This implies that when
the container is no longer necessary (scale down), it can be safely deleted as no valuable data is
stored within the container. A special command line flag is present for this purpose, such that a
container is automatically deleted upon container shutdown. Using the following command, such
a container can be instantiated and the entrypoint will be launched automatically:
podman run --rm --name mycontainer testcontainer
As you can see the testcontainer image is used to create a container named mycontainer. You
will note that the container entrypoint process remains in the foreground. To avoid this, add the
-d command line flag.
The currently active containers can be inspected using the command podman ps. Now, how
can we access the Python process running on the internal container network? We need to retrieve
the internal IP address. To do so, you can use the command:
podman container inspect mycontainer
This gives a lot of information about the container. Search for IPAddress to obtain the internal
IP address. Through this IP, you should be able to access the Python webserver on port 8000.
You can also map the container’s port to the host port using the -p option. But note that in the
context of this assignment, you will only want to expose the port of the load balancer, the ports
of the Web application instances remain internal!
Finally, to stop a container you can use: podman stop.
B.4 Mounting a directory within a container
For the Web application you need to ensure that all containers have access to the same directory
in which the objects are stored. First make sure you have created such a directory on the host
(we use /srv/objects). In your container image, you want to create a mountpoint, for example
/objects. With this in place, add the following command line option to the podman run command
to mount the host directory /srv/objects within the container:
-v /srv/objects:/objects
C Suggestions on the load balancer
You may write your own load balancer or use an existing one such as HAProxy. As you may have
read above, Alpine does provide HAProxy packages. Therefore, it is relatively easy to set up a
container in which you can run the load balancer.
HAProxy allows you to read out statistics via HTTP. To enable this, you need to add a section
such as the following to the configuration file:
frontend stats
bind *:9999
stats enable
stats uri /stats
stats refresh 1s
7
After restarting HAProxy, you can connect to port 9999 to read statistics. For instance, con￾nect to http://10.0.3.6:9999/stats (of course replace the IP address with the correct one).
The following URL returns machine-readable CSV output, which will be useful for your scaling
controller: http://10.0.3.6:9999/stats;csv.
D Considerations for the scaling controller
The scaling controller consists of two parts that can be programmed independently (make good
use of your team’s resources) before these are integrated. The monitoring part should monitor the
load balancer and/or the container instances running the web application. It needs to retrieve the
information required to make scaling decisions (should we scale up, scale down? And if so by how
many instances?). Note that this refers to the Monitoring, Analysis and Planning phases of the
MAPE feedback loop. If you chose to use HAProxy, you can monitor the haproxy daemon which
has an option to provide you with statistics
The podman part needs to be capable of creating new container instances, stopping instances,
listing all instances (and their IP addresses) and all other container management utilities you
need in order to make the scaling controller work. This is the (final) Execution phase of the
MAPE loop. podman commands or API calls might be blocking by default, in which case you
want to look into asynchronous calls or multi-threading in order to make your scaling controller
more responsive. This way, you can continue monitoring, while the podman commands are being
executed in a different thread.
After instructing podman to start or stop a container, you also need to update the configuration
of the load balancer. How exactly this should be done depends on the load balancer have you
chosen to use. In the case of HAProxy there is no clear runtime API to update the list of
servers. The most straightforward way to achieve this is to have your scaling controller regenerate
the HAProxy configuration file, send this configuration to the load balancer container (or use a
volume mount?) and reload HAProxy. Hacky, but it works and it appears this is used in practice
(!).
podman can be controlled through shell commands, but also via its RESTful API. This API
is documented in detail: https://docs.podman.io/en/v3.2.3/_static/api.html.
While you can target this RESTful API directly, fortunately also language bindings exist for at
least Python and Go. We give a small example of the Python API. Before you can use this API,
the module needs to be installed with pip3 install podman.
Obtain a list of names of defined containers:
from podman import PodmanClient
client = PodmanClient(base_url="unix:///run/podman/podman.sock")
l = [c.name for c in client.containers.list()]
Get a handle on a container and if it is running print the IP addresses of this container3
:
from podman import PodmanClient
client = PodmanClient(base_url="unix:///run/podman/podman.sock")
c = client.containers.get("testcontainer")
if c.status == ’running’:
print(c.attrs[’NetworkSettings’][’Networks’][’podman’][’IPAddress’])
Containers can be stopped with the .stop() method. The method .wait(condition=’running’)
waits (blocking) until a container is running.
3The podman in the dictionary access is the name of the default container network, see the command podman
network ls.
8
The Python module has extensive documentation, use the help command in an interactive Python
shell. We could not find this documentation online.
E Testing the system
In order to test the completed system, you want to use a HTTP load generator. Options are
writing such a generator yourself, jMeter, Locust or something else you find to be suitable. For
your experiments it is important to first design your experiments and after that decide on a load or
traffic pattern generator to use. You do not want to be limited in your experiments by a previously
selected load generator.
Locust is Python-based and works as a command-line utility. First, you need to write a locust file
(refer to the website https://docs.locust.io/en/stable/quickstart.html for an example).
After that you can start locust:
locust -f mylocustfile.py --headless -u 10 -t 300s -r 0.5
Where -u configures the number of (concurrent) users, -t configures the runtime of the experiment
and -r configures the rate in which clients are spawned. The values for the parameters that are
given are just examples, you should set up your own experiment.
請加QQ：99515681  郵箱：99515681@qq.com   WX：codehelp