Skip to content

Commit 689ffea

Browse files
adeepkit01Viyom
adeepkit01
authored and
Viyom
committed
AQM Evaluation Suite module
1 parent 60cfa6d commit 689ffea

30 files changed

+3332
-0
lines changed

‎src/aqm-eval-suite/doc/aqm-eval-suite.rst‎

Lines changed: 168 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,168 @@
1+
AQM Evaluation Suite
2+
--------------------
3+
4+
.. include:: replace.txt
5+
.. highlight:: cpp
6+
7+
.. heading hierarchy:
8+
------------- Chapter
9+
************* Section (#.#)
10+
============= Subsection (#.#.#)
11+
############# Paragraph (no number)
12+
13+
14+
The AQM Evaluation Suite is an automated framework for comparing the performance of
15+
ns-3 queue disciplines based on the scenarios mentioned in RFC 7928. It includes
16+
simulation setup, topology creation, traffic generation, program execution, results
17+
collection and their graphical representation.
18+
19+
20+
Introducing the AQM Evaluation Suite
21+
************************************
22+
23+
AQM Evaluation Suite automates the cycle of simulation setup to results collection
24+
in ns-3. It closely follows the recommendations provided in RFC 7928 to setup the
25+
simulation scenarios and collects results in the suggested formats. A user can
26+
choose to either run an individual scenario from the set of scenarios provided in
27+
the suite, or run all scenarios at once. Results for each scenario are systematically
28+
stored in text and graphical formats.
29+
30+
The following provides more details about the architecture of the suite, users
31+
interaction with the suite, its scope and limitations, and steps to extend the
32+
suite for evaluating new AQM algorithms.
33+
34+
Architecture of AQM Evaluation Suite
35+
************************************
36+
37+
The suite is implemented in ``src/aqm-eval-suite`` directory.
38+
39+
Model
40+
=====
41+
42+
``src/aqm-eval-suite/model`` contains an implementation of the following three
43+
primary classes:
44+
45+
* class :cpp:class:`EvaluationTopology`: This class has three major functionalities:
46+
47+
* Creating the topology: It sets up a point-to-point dumbbell topology by using
48+
:cpp:class:`PointToPointDumbbellHelper`: with required number of nodes, and
49+
configures the data rate and for all the links. It also installs the desired queue
50+
discipline on the router node.
51+
52+
* Installing application on nodes: it provides an API for configuration of
53+
applications. This API takes the application parameters such as data rate,
54+
packet size, transport protocol, initial congestion window in case of TCP,
55+
maximum bandwidth and one way delay of the channels.
56+
57+
* Getting the metrics of interest from the experiment: It uses the traces
58+
sources provided by different classes of ns-3 for the calculation of metrics.
59+
The metrics recommended in the RFC are queue-delay, goodput, throughput and
60+
number of drops.
61+
62+
* class :cpp:class:`EvalApp`: This class is based on :cpp:class:`OnOffApplication`:
63+
and is used for generating TCP and UDP traffic in the suite. The native class
64+
:cpp:class:`OnOffApplication`: in ns-3 creates sockets at the start time of an
65+
application. Thus, to configure different values for parameters like initial
66+
congestion window in TCP is non-trivial, since they cannot be configured before
67+
the socket is created and after the application starts. To overcome this,
68+
:cpp:class:`EvalApp`: is implemented on the same principles as that of the
69+
:cpp:class:`OnOffApplication`: in which a socket is created and the application
70+
is started only after its parameters are configured.
71+
72+
* class :cpp:class:`EvalTimestampTag`: This is a subclass of :cpp:class:`Tag`: and
73+
has been developed to fetch the queue delay information from :cpp:class:`QueueDisc`:.
74+
When a packet is enqueued by the QueueDisc, this tag is added with a timestamp
75+
(as the enqueue time) and when the packet is dequeued, the queue delay is computed
76+
as the difference between the dequeue time and the enqueue time.
77+
78+
Helper
79+
======
80+
81+
``src/aqm-eval-suite/helper`` contains an implementation of the following class:
82+
83+
* class :cpp:class:`ScenarioImpl`: This class implements the following two methods:
84+
85+
* ``ScenarioImpl::CreateScenario ()``: This is a virtual function implemented by
86+
each scenario according to the topology and traffic profiles mentioned in the RFC.
87+
88+
* ``ScenarioImpl::RunSimulation ()``: This method takes the scenario created by
89+
each subclass and runs them with all the queue disciplines available in ns-3.
90+
91+
Utils
92+
=====
93+
94+
``src/aqm-eval-suite/utils`` directory provides four Python scripts that take
95+
performance metrics computed in the suite as input, and generate a graph with
96+
Queuing Delay as the X-axis against Goodput as the Y-axis. The graph depicts an
97+
ellipse which is plotted as per the guidelines mentioned in the RFC and [Remy].
98+
The co-variance between the queuing delay and goodput is determined by the
99+
orientation of the ellipse, and helps to analyze the effect of traffic load on
100+
Goodput and Queuing Delay.
101+
102+
Examples
103+
========
104+
105+
``src/aqm-eval-suite/examples`` directory provides a set of programs, each
106+
corresponding to a specific scenario listed in RFC 7928. Each program can be
107+
run individually. Alternatively, `aqm-eval-suite-runner.cc` allows the user
108+
to run all scenarios at once.
109+
110+
User interaction with the suite
111+
*******************************
112+
113+
Users can learn about the list of scenarios available in the suite from
114+
``src/aqm-eval/examples`` directory. The programs can be run in a usual way.
115+
For example, ``aggressive-transport-sender.cc`` is equivalent to the scenario
116+
described in Section 5.2 of the RFC. Assuming examples have been enabled
117+
during configure, the following commands would run ``aggressive-transport-sender.cc``
118+
119+
::
120+
121+
$ ./waf --run "aqm-eval-suite-runner --number=5.2"
122+
123+
or
124+
125+
::
126+
127+
$ ./waf --run "aqm-eval-suite-runner --name=AggressiveTransportSender"
128+
129+
To run all scenarios at once, the following command could be used:
130+
131+
::
132+
133+
$ ./waf --run "aqm-eval-suite-runner --name=All"
134+
135+
Simulating additional AQM algorithms using this suite
136+
*****************************************************
137+
138+
* By default, the suite evaluates AQM algorithms implemented in |ns3|. To
139+
simulate additional AQM algorithms, such as the ones designed by the user,
140+
the ``addAQM`` method of ``ScenarioImpl`` can be used in the scenarios
141+
available in ``src/aqm-eval-suite/examples``. For example, to add a new AQM
142+
of typeId ``ns3::ExampleQueueDisc`` in ``aggressive-transport-sender.cc``,
143+
``CreateScenario`` method can be modified as shown in the code below:
144+
145+
.. code-block:: c++
146+
147+
EvaluationTopology
148+
AggressiveTransportSender::CreateScenario (std::string aqm)
149+
{
150+
.
151+
.
152+
addAQM ("ns3::ExampleQueueDisc");
153+
EvaluationTopology et ("AggressiveTransportSender", nflow, pointToPoint, aqm, 1460);
154+
.
155+
.
156+
}
157+
158+
Scope and limitations of the suite
159+
**********************************
160+
161+
* All scenarios described in Section 5, 6 and 8 of RFC 7928 are supported.
162+
163+
* Scenarios listed in Section 7 and 9 are not yet supported.
164+
165+
* Currently, the suite cannot be used to study the interaction of queue disciplines
166+
with Explicit Congestion Notification (ECN) and Scheduling Algorithms.
167+
168+
* Multi-AQM scenarios are not yet supported.

‎src/aqm-eval-suite/examples/LBE-transport-sender.cc‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
2+
/*
3+
* Copyright (c) 2017 NITK Surathkal
4+
*
5+
* This program is free software; you can redistribute it and/or modify
6+
* it under the terms of the GNU General Public License version 2 as
7+
* published by the Free Software Foundation;
8+
*
9+
* This program is distributed in the hope that it will be useful,
10+
* but WITHOUT ANY WARRANTY; without even the implied warranty of
11+
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
12+
* GNU General Public License for more details.
13+
*
14+
* You should have received a copy of the GNU General Public License
15+
* along with this program; if not, write to the Free Software
16+
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
17+
*
18+
* Authors: Ankit Deepak <adadeepak8@gmail.com>
19+
* Shravya K. S. <shravya.ks0@gmail.com>
20+
* Mohit P. Tahiliani <tahiliani@nitk.edu.in>
21+
*/
22+
23+
/*
24+
* This example is equivalent to the scenario described in Section 5.1.2
25+
* of RFC 7928 (https://tools.ietf.org/html/rfc7928#section-5.1).
26+
*/
27+
28+
#include "ns3/log.h"
29+
#include "ns3/simulator.h"
30+
#include "ns3/aqm-eval-suite-module.h"
31+
32+
using namespace ns3;
33+
34+
NS_LOG_COMPONENT_DEFINE ("LbeTransportSender");
35+
36+
class LbeTransportSender : public ScenarioImpl
37+
{
38+
public:
39+
LbeTransportSender ();
40+
~LbeTransportSender ();
41+
42+
protected:
43+
virtual EvaluationTopology CreateScenario (std::string aqm);
44+
};
45+
46+
LbeTransportSender::LbeTransportSender ()
47+
{
48+
}
49+
50+
LbeTransportSender::~LbeTransportSender ()
51+
{
52+
}
53+
54+
EvaluationTopology
55+
LbeTransportSender::CreateScenario (std::string aqm)
56+
{
57+
PointToPointHelper pointToPoint;
58+
pointToPoint.SetDeviceAttribute ("DataRate", StringValue ("1Mbps"));
59+
pointToPoint.SetChannelAttribute ("Delay", StringValue ("48ms"));
60+
uint32_t nflow = 2;
61+
62+
EvaluationTopology et ("LbeTransportSender", nflow, pointToPoint, aqm, 1460);
63+
ApplicationContainer ac1 = et.CreateFlow (StringValue ("1ms"),
64+
StringValue ("1ms"),
65+
StringValue ("10Mbps"),
66+
StringValue ("10Mbps"),
67+
"ns3::TcpLedbat", 0, DataRate ("10Mb/s"), 3);
68+
69+
ApplicationContainer ac2 = et.CreateFlow (StringValue ("1ms"),
70+
StringValue ("1ms"),
71+
StringValue ("10Mbps"),
72+
StringValue ("10Mbps"),
73+
"ns3::TcpNewReno", 0, DataRate ("10Mb/s"), 3);
74+
75+
ac1.Start (Seconds (0));
76+
ac1.Stop (Seconds (300));
77+
78+
ac2.Start (Seconds (0));
79+
ac2.Stop (Seconds (300));
80+
return et;
81+
}
82+
83+
int
84+
main (int argc, char *argv[])
85+
{
86+
CommandLine cmd;
87+
cmd.Parse (argc, argv);
88+
89+
LbeTransportSender sce;
90+
sce.ConfigureQueueDisc (45, 1460, "1Mbps", "48ms");
91+
sce.RunSimulation (Seconds (310));
92+
}

‎src/aqm-eval-suite/examples/aggressive-transport-sender.cc‎

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
2+
/*
3+
* Copyright (c) 2017 NITK Surathkal
4+
*
5+
* This program is free software; you can redistribute it and/or modify
6+
* it under the terms of the GNU General Public License version 2 as
7+
* published by the Free Software Foundation;
8+
*
9+
* This program is distributed in the hope that it will be useful,
10+
* but WITHOUT ANY WARRANTY; without even the implied warranty of
11+
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
12+
* GNU General Public License for more details.
13+
*
14+
* You should have received a copy of the GNU General Public License
15+
* along with this program; if not, write to the Free Software
16+
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
17+
*
18+
* Authors: Ankit Deepak <adadeepak8@gmail.com>
19+
* Shravya K. S. <shravya.ks0@gmail.com>
20+
* Mohit P. Tahiliani <tahiliani@nitk.edu.in>
21+
*/
22+
23+
/*
24+
* This example is equivalent to the scenario described in Section 5.2
25+
* of RFC 7928 (https://tools.ietf.org/html/rfc7928#section-5.2).
26+
*
27+
* Note: BIC TCP is used as an aggressive sender due to unavailability
28+
* of CUBIC TCP module in ns-3.
29+
*/
30+
31+
#include "ns3/log.h"
32+
#include "ns3/simulator.h"
33+
#include "ns3/aqm-eval-suite-module.h"
34+
35+
using namespace ns3;
36+
37+
class AggressiveTransportSender : public ScenarioImpl
38+
{
39+
public:
40+
AggressiveTransportSender ();
41+
~AggressiveTransportSender ();
42+
43+
protected:
44+
virtual EvaluationTopology CreateScenario (std::string aqm);
45+
};
46+
47+
AggressiveTransportSender::AggressiveTransportSender ()
48+
{
49+
}
50+
51+
AggressiveTransportSender::~AggressiveTransportSender ()
52+
{
53+
}
54+
55+
EvaluationTopology
56+
AggressiveTransportSender::CreateScenario (std::string aqm)
57+
{
58+
PointToPointHelper pointToPoint;
59+
pointToPoint.SetDeviceAttribute ("DataRate", StringValue ("1Mbps"));
60+
pointToPoint.SetChannelAttribute ("Delay", StringValue ("48ms"));
61+
uint32_t nflow = 1;
62+
63+
EvaluationTopology et ("AggressiveTransportSender", nflow, pointToPoint, aqm, 1460);
64+
ApplicationContainer ac = et.CreateFlow (StringValue ("1ms"),
65+
StringValue ("1ms"),
66+
StringValue ("10Mbps"),
67+
StringValue ("10Mbps"),
68+
"ns3::TcpBic", 0, DataRate ("10Mb/s"), 3);
69+
70+
ac.Start (Seconds (0));
71+
ac.Stop (Seconds (300));
72+
return et;
73+
}
74+
75+
int
76+
main (int argc, char *argv[])
77+
{
78+
CommandLine cmd;
79+
cmd.Parse (argc, argv);
80+
81+
AggressiveTransportSender sce;
82+
sce.ConfigureQueueDisc (45, 1460, "1Mbps", "48ms");
83+
sce.RunSimulation (Seconds (310));
84+
}

0 commit comments

Comments
 (0)