Can mOS API run on a Raspberry Pi B+?

[VuXuanPhong]

Hi,

I am a student from Viet Nam. I have a project that build a IDS/IPS with mOS API on Raspberry Pi B+.

I try to build mOS API on Raspbian Lite aarch 64, and then got an error: "unrecognized command line option -m64".

In my point, mOS API can't build on Raspberry Pi machine, but i want to confirm that.

Please tell me can mOS API build on Raspberry Pi?

Sorry about my bad English.

Thanks.

[ajamshed]

Hi @VuXuanPhong,

-m64 is an architecture specific gcc switch. The setup.sh script was, in fact, written for Linux/Intel-based environments (x86_64-native-linuxapp-gcc, please see this link for more details). However one can build mOS networking stack on a a Raspberry device (without directly using setup.sh script file) as long as its depending libraries can compile in your platform. Can you try building DPDK first and see if it compiles without any issues? You can also try running a sample dpdk application (e.g. l3fwd) to make sure everything checks out fine.

Once you have verified DPDK's compatibility, we can discuss how you can compile mOS in your setup.

Also, we are in the process of revamping (#20) setup.sh to make it more general (so that it can transparently work for various platforms). Please reach back once you have tested DPDK.

[VuXuanPhong]

Thank you so much. I will build DPDK and run an application base on DPDK.

[VuXuanPhong]

Hi,
I am trying to build DPDK on a Raspberry Pi with Raspbean OS Lite 64bit, but I did not get DPDK compiled. I plan to try on openSUSE OS as I found a tutorial that compiles DPDK on Pi. So I will come back later to confirm whether it can be doen or not.

I just want to ask you questions about mOS API Library to make a plan. I read mOS guide and mOS paper many time, and I think that I understand how mOS API work. My Teacher want to make an intrusion detection system base on mOS. I also read about mSnort in mOS paper, my goal is to combine Snort with mOS such as mSnort.

As far as I understand, mSnort transforms the signatures-rule of Snort into event-action pairs in mOS, so this becomes event chains. Each function in an event chain is both the filter function and the handler function. Two important functions are pattern matching function and PCRE matching function. Is these function borrowed from Snort? And pattern matching function is a multi-event filter function, this mean that when a string matches a pattern, a event will be triggered. How can I make it while a filter-function can only return true or false?

As I read in the mOS paper, mSnort use a parse function to transforms Snort rule to event-handle pair. What does this function do? Does this function read a file containing a set of rule line by line and parse each rule to an event chain?

Is the thing I said above wrong?

Wish you have a nice day.

Thank you.

[ajamshed]

@VuXuanPhong,

We modified Snort3 to integrate with mOS stack. We reused Snort's own pattern matching and PCRE routines for this purpose. Both functions were used as filters. We used a combination of mtcp_alloc_event(), mtcp_define_event(), and mtcp_raise_event() to create a multi-event filter. By using a multi-event filter, you can get different types of events depending on the event id (which is the last argument of the callback function). Each unique event represents a different signature match (as evaluated from the Aho-Corasick pattern matching routine in Snort). You can refer to the man pages for details: http://mos.kaist.edu/index_man.html.

Your last few questions are quite broad. Very briefly, we used an initialization routine that read all rules from a rule file and created filter-event-handler functions. I suggest that you go through a tutorial on how a signature-based IDS usually works for further clarification. A good starting point can be section 2 of our kargus paper.

[VuXuanPhong]

Thank for your reply message. I will study according to your instructions.

[VuXuanPhong]

Hello @ajamshed.
I read the Kargus paper and studied about Snort in this paper https://drive.google.com/open?id=0BwOhgrRb0pbfY3Q3OS01eWtwMjljdXZIWnpCejBZY0lpak9r. Snort maintain a set of rules to examine whether a flow contains malicious packets. The rules (or signature) consist of a large number of rule options that express various attack patterns (e.g., content, pcre), payload type (e.g., http_header) and conditions (e.g., only_stream and to_- server) as mentioned in mSnort. And your team express each rule option type as a UDE filter, and synthesize each rule as a chain of UDEs. I understand the basic logic above but I still have some questions in my mind which I can't answer myself. In the example in mOS paper:

According to my understanding:

• e1, e2 events will be triggered in the sender stack whenever a packet arrives and e3, e4 events will be triggered whenever enough flow data is reassembled.
• Why need two pattern matching functions FTAC1 and FTAC2? FTAC1 is used when the rule does not indicate the payload type of packet?
• Is FTDIR function also multi-event filter function? Is FTDIR equivalent to flow keyword in rules?
• The packet structure in Snort and mOS is different, then how can you reuse pattern matching function in Snort. If possible, can you indicate for me the pattern matching function and pcre function in Snort 's source code? Just which source file to research. Please. The source code of Snort is too complex for me.

I can't figure it out when you say that "read all rules from a rule file and created filter-event-handler functions". When I read mOS paper, I understand that: mSnort express each rule option type as a UDE filter and in mSnort, your team implemented 17 rule options of Snort rule. In my point, your team write 17 filter function correspond to 17 rule options. And then read rule file line by line, process each line, when a rule option is related, call the corresponding function as filter-handle function. Please tell me more detail about the initialization routine in mSnort.
Currently, I am a full-time intern, I don't have enough time to research in this project. I only have myself in this project, so I can only ask you for help.
And how many time your team need to complete mSnort?
Thank you so much.

[VuXuanPhong]

I just want to test mOS event system, so I write a program base on mOS that trigger an event when a ICMP ping detected.
`
#define _LARGEFILE64_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <stdint.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <fcntl.h>
#include <dirent.h>
#include <string.h>
#include <time.h>
#include <pthread.h>
#include <signal.h>
#include <linux/if_ether.h>
#include <linux/tcp.h>
#include <mos_api.h>
#include <ctype.h>
#include "cpu.h"
#include "http_parsing.h"
#include "debug.h"
#include "applib.h"
/----------------------------------------------------------------------------/
/* default configuration file path */
#define MOS_CONFIG_FILE "config/mos.conf"

#define EXIT_WITH_ERROR(f, m...) {
fprintf(stderr, "[%10s:%4d] errno: %u" f, FUNCTION, LINE, errno, ##m);
exit(EXIT_FAILURE);
}

struct thread_context
{
mctx_t mctx; /* per-thread mos context /
int mon_listener; / listening socket for flow monitoring */
};

static void
ApplyActionPerFlow(mctx_t mctx, int msock, int side,
uint64_t events, filter_arg_t *arg)

{
/* this function is called at the first SYN */
struct pkt_info p;

if (mtcp_getlastpkt(mctx, msock, side, &p) < 0)
	EXIT_WITH_ERROR("Failed to get packet context!\n");

printf("A ping \n");

}

static bool
CatchICMP(mctx_t mctx, int sockid,
int side, uint64_t events, filter_arg_t *arg)
{
struct pkt_info p;

if (mtcp_getlastpkt(mctx, sockid, side, &p) < 0)
	EXIT_WITH_ERROR("Failed to get packet context!!!\n");

return (p.iph->protocol == 1);

}

static void
cb_printstat(mctx_t mctx, int sock, int side,
uint64_t events, filter_arg_t *arg)
{
printf("Runing...\n");
}

static void
CreateAndInitThreadContext(struct thread_context* ctx,
int core, event_t icmp_event)
{
struct timeval tv_1sec = { /* 1 second */
.tv_sec = 1,
.tv_usec = 0
};

ctx->mctx = mtcp_create_context(core);

/* create socket  */
ctx->mon_listener = mtcp_socket(ctx->mctx, AF_INET,
				MOS_SOCK_MONITOR_STREAM, 0);
if (ctx->mon_listener < 0)
	EXIT_WITH_ERROR("Failed to create monitor listening socket!\n");

/* register callback */
if (mtcp_register_callback(ctx->mctx, ctx->mon_listener,
						   icmp_event,
						   MOS_NULL,
			   			   ApplyActionPerFlow) == -1)
	EXIT_WITH_ERROR("Failed to register callback func!\n");

/* CPU 0 is in charge of printing stats */
if (ctx->mctx->cpu == 0 &&
	mtcp_settimer(ctx->mctx, ctx->mon_listener,
				  &tv_1sec, cb_printstat))
	EXIT_WITH_ERROR("Failed to register timer callback func!\n");

}
/----------------------------------------------------------------------------/
static void
WaitAndCleanupThreadContext(struct thread_context* ctx)
{
/* wait for the TCP thread to finish */
mtcp_app_join(ctx->mctx);

/* close the monitoring socket */
mtcp_close(ctx->mctx, ctx->mon_listener);

/* tear down */
mtcp_destroy_context(ctx->mctx);	

}
/----------------------------------------------------------------------------/
int
main(int argc, char **argv)
{
int ret, i;
char fname = MOS_CONFIG_FILE; / path to the default mos config file /
struct mtcp_conf mcfg;
struct thread_context ctx[MAX_CPUS] = {{0}}; / init all fields to 0 */
event_t icmp_event;
int num_cpus;
int opt, rc;

/* get the total # of cpu cores */
num_cpus = GetNumCPUs();

while ((opt = getopt(argc, argv, "c:n:")) != -1) {
	switch (opt) {
		case 'c':
			fname = optarg;
			break;
		case 'n':
			if ((rc=atoi(optarg)) > num_cpus) {
				EXIT_WITH_ERROR("Available number of CPU cores is %d "
						"while requested cores is %d\n",
						num_cpus, rc);
			}
			num_cpus = rc;
			break;
		default:
			printf("Usage: %s [-c mos_config_file] "
			       "[-f simple_firewall_config_file]\n", 
			       argv[0]);
			return 0;
	}
}

/* parse mos configuration file */
ret = mtcp_init(fname);
if (ret)
	EXIT_WITH_ERROR("Failed to initialize mtcp.\n");

/* set the core limit */
mtcp_getconf(&mcfg);
mcfg.num_cores = num_cpus;
mtcp_setconf(&mcfg);

mtcp_getconf(&mcfg);
// ICMP event
icmp_event = mtcp_define_event(MOS_ON_ORPHAN, CatchICMP, NULL);
if (icmp_event == MOS_NULL_EVENT)
	EXIT_WITH_ERROR("mtcp_define_event() failed!");

for (i = 0; i < mcfg.num_cores; i++)
	CreateAndInitThreadContext(&ctx[i], i, icmp_event);

for (i = 0; i < mcfg.num_cores; i++) {
	WaitAndCleanupThreadContext(&ctx[i]);
  	TRACE_INFO("Message test thread %d joined.\n", i);	  
}	

mtcp_destroy();

return EXIT_SUCCESS;

}
`
icmp_event is created with the MOS_ON_ORPHAN base event and the filter CatchICMP function. As my expectation, the program will print "A ping" when I ping from the other laptop to. But nothing happen.

I tested on a desktop machine with 16 Gb ram and 4 core cpu. The program run in passive mode with pcap.

Please tell me what i am wrong.
Your answer will be references for me.
Thank you so much.

[ajamshed]

Hello @ajamshed.
I read the Kargus paper and studied about Snort in this paper https://drive.google.com/open?id=0BwOhgrRb0pbfY3Q3OS01eWtwMjljdXZIWnpCejBZY0lpak9r. Snort maintain a set of rules to examine whether a flow contains malicious packets. The rules (or signature) consist of a large number of rule options that express various attack patterns (e.g., content, pcre), payload type (e.g., http_header) and conditions (e.g., only_stream and to_- server) as mentioned in mSnort. And your team express each rule option type as a UDE filter, and synthesize each rule as a chain of UDEs. I understand the basic logic above but I still have some questions in my mind which I can't answer myself. In the example in mOS paper:

According to my understanding:

• e1, e2 events will be triggered in the sender stack whenever a packet arrives and e3, e4 events will be triggered whenever enough flow data is reassembled.
• Why need two pattern matching functions FTAC1 and FTAC2? FTAC1 is used when the rule does not indicate the payload type of packet?
• Is FTDIR function also multi-event filter function? Is FTDIR equivalent to flow keyword in rules?
• The packet structure in Snort and mOS is different, then how can you reuse pattern matching function in Snort. If possible, can you indicate for me the pattern matching function and pcre function in Snort 's source code? Just which source file to research. Please. The source code of Snort is too complex for me.

I can't figure it out when you say that "read all rules from a rule file and created filter-event-handler functions". When I read mOS paper, I understand that: mSnort express each rule option type as a UDE filter and in mSnort, your team implemented 17 rule options of Snort rule. In my point, your team write 17 filter function correspond to 17 rule options. And then read rule file line by line, process each line, when a rule option is related, call the corresponding function as filter-handle function. Please tell me more detail about the initialization routine in mSnort.
Currently, I am a full-time intern, I don't have enough time to research in this project. I only have myself in this project, so I can only ask you for help.
And how many time your team need to complete mSnort?
Thank you so much.

Hi @VuXuanPhong.

You asked a lot of questions in this message. I will try to answer all of them here.
[Correction] e3 is triggered when mOS detects a connection termination.

[Correction] FT-AC1 & FT-AC2 are both Aho-Corasick-based (AC) multi-pattern matching filter functions that scan for suspect malicious strings in payloads. Snort's pattern matching module has evolved over time. Different AC DFAs are now, in fact, initialized based on various types (tcp-payload, http-payload, smtp-payload, etc.) In the example figure above, we show two AC filters above that correspond to tcp-payload and http-payload. This means that Snort's detection engine runs the packet/stream instance across both (in sequence) if the flow is http. i hope this makes sense.

[Correction] FTDIR is a uni-event filter (default version) and it evaluates the following Snort sub-rule-option flow: to_server or flow:to_client.

We carefully typecasted mOS-based packet data-structure to Snort's. We could not keep one common packet data structure, otherwise we would have ended up modifying the Snort source code a lot.

I understand that the Snort source code is a bit complicated. I suggest that you spend a few days understanding how the IDS is designed. I can't pinpoint the exact file that you should be focusing on since there is a huge leap in code structure from Snort 2.9.x (C-based, single-threaded) to Snort 3.x (C++-based, multi-threaded). Which version are you analyzing?

You pretty much captured everything related to mSnort initialization. We had to modify the Snort init and rule_parser modules to integrate our mOS-specific changes. I suggest that you do an in-depth study of the 2 modules.

Making changes to Snort took a few days. But we had past experience with Snort source code development. For a newbie, I am afraid it may take more than a few weeks to get familiarized with the source code. I suggest that you first pick a version (2.9.x or 3.x). Snort-.2.9.x is simpler to study than Snort-3.x. You can probably start with that.

[ajamshed]

I just want to test mOS event system, so I write a program base on mOS that trigger an event when a ICMP ping detected.
`
#define LARGEFILE64_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <stdint.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <fcntl.h>
#include <dirent.h>
#include <string.h>
#include <time.h>
#include <pthread.h>
#include <signal.h>
#include <linux/if_ether.h>
#include <linux/tcp.h>
#include <mos_api.h>
#include <ctype.h>
#include "cpu.h"
#include "http_parsing.h"
#include "debug.h"
#include "applib.h"
/----------------------------------------------------------------------------_/
/* default configuration file path */
#define MOS_CONFIG_FILE "config/mos.conf"

#define EXIT_WITH_ERROR(f, m...) {
fprintf(stderr, "[%10s:%4d] errno: %u" f, FUNCTION, LINE, errno, ##m);
exit(EXIT_FAILURE);
}

struct thread_context
{
mctx_t mctx; /* per-thread mos context / int mon_listener; / listening socket for flow monitoring */
};

static void
ApplyActionPerFlow(mctx_t mctx, int msock, int side,
uint64_t events, filter_arg_t *arg)

{
/* this function is called at the first SYN */
struct pkt_info p;

if (mtcp_getlastpkt(mctx, msock, side, &p) < 0)
	EXIT_WITH_ERROR("Failed to get packet context!\n");

printf("A ping \n");

}

static bool
CatchICMP(mctx_t mctx, int sockid,
int side, uint64_t events, filter_arg_t *arg)
{
struct pkt_info p;

if (mtcp_getlastpkt(mctx, sockid, side, &p) < 0)
	EXIT_WITH_ERROR("Failed to get packet context!!!\n");

return (p.iph->protocol == 1);

}

static void
cb_printstat(mctx_t mctx, int sock, int side,
uint64_t events, filter_arg_t *arg)
{
printf("Runing...\n");
}

static void
CreateAndInitThreadContext(struct thread_context* ctx,
int core, event_t icmp_event)
{
struct timeval tv_1sec = { /* 1 second */
.tv_sec = 1,
.tv_usec = 0
};

ctx->mctx = mtcp_create_context(core);

/* create socket  */
ctx->mon_listener = mtcp_socket(ctx->mctx, AF_INET,
				MOS_SOCK_MONITOR_STREAM, 0);
if (ctx->mon_listener < 0)
	EXIT_WITH_ERROR("Failed to create monitor listening socket!\n");

/* register callback */
if (mtcp_register_callback(ctx->mctx, ctx->mon_listener,
						   icmp_event,
						   MOS_NULL,
			   			   ApplyActionPerFlow) == -1)
	EXIT_WITH_ERROR("Failed to register callback func!\n");

/* CPU 0 is in charge of printing stats */
if (ctx->mctx->cpu == 0 &&
	mtcp_settimer(ctx->mctx, ctx->mon_listener,
				  &tv_1sec, cb_printstat))
	EXIT_WITH_ERROR("Failed to register timer callback func!\n");

}
/----------------------------------------------------------------------------/
static void
WaitAndCleanupThreadContext(struct thread_context* ctx)
{
/* wait for the TCP thread to finish */
mtcp_app_join(ctx->mctx);

/* close the monitoring socket */
mtcp_close(ctx->mctx, ctx->mon_listener);

/* tear down */
mtcp_destroy_context(ctx->mctx);	

}
/----------------------------------------------------------------------------/
int
main(int argc, char **argv)
{
int ret, i;
char fname = MOS_CONFIG_FILE; / path to the default mos config file / struct mtcp_conf mcfg; struct thread_context ctx[MAX_CPUS] = {{0}}; / init all fields to 0 */
event_t icmp_event;
int num_cpus;
int opt, rc;

/* get the total # of cpu cores */
num_cpus = GetNumCPUs();

while ((opt = getopt(argc, argv, "c:n:")) != -1) {
	switch (opt) {
		case 'c':
			fname = optarg;
			break;
		case 'n':
			if ((rc=atoi(optarg)) > num_cpus) {
				EXIT_WITH_ERROR("Available number of CPU cores is %d "
						"while requested cores is %d\n",
						num_cpus, rc);
			}
			num_cpus = rc;
			break;
		default:
			printf("Usage: %s [-c mos_config_file] "
			       "[-f simple_firewall_config_file]\n", 
			       argv[0]);
			return 0;
	}
}

/* parse mos configuration file */
ret = mtcp_init(fname);
if (ret)
	EXIT_WITH_ERROR("Failed to initialize mtcp.\n");

/* set the core limit */
mtcp_getconf(&mcfg);
mcfg.num_cores = num_cpus;
mtcp_setconf(&mcfg);

mtcp_getconf(&mcfg);
// ICMP event
icmp_event = mtcp_define_event(MOS_ON_ORPHAN, CatchICMP, NULL);
if (icmp_event == MOS_NULL_EVENT)
	EXIT_WITH_ERROR("mtcp_define_event() failed!");

for (i = 0; i < mcfg.num_cores; i++)
	CreateAndInitThreadContext(&ctx[i], i, icmp_event);

for (i = 0; i < mcfg.num_cores; i++) {
	WaitAndCleanupThreadContext(&ctx[i]);
  	TRACE_INFO("Message test thread %d joined.\n", i);	  
}	

mtcp_destroy();

return EXIT_SUCCESS;

}
`
icmp_event is created with the MOS_ON_ORPHAN base event and the filter CatchICMP function. As my expectation, the program will print "A ping" when I ping from the other laptop to. But nothing happen.

I tested on a desktop machine with 16 Gb ram and 4 core cpu. The program run in passive mode with pcap.

Please tell me what i am wrong.
Your answer will be references for me.
Thank you so much.

Wow. You posted the full source code. @VuXuanPhong, it will be difficult to analyze and test the entire source code as I am a bit busy. I can give you a few pointers where exactly the problems may lie.

1- Did you go through the documentation (http://mos.kaist.edu/guide/)? This page indicates that you can't run PCAP mode on multi-cores since our pcap plugin does not split traffic across multi-cores. Also, did you put the forward variable in mos.conf equal to 0 (as mentioned in this page). When you are running mOS app in passive mode, you should not be forwarding packet. It looks like packets are trying to be sent out (ICMPOutput line).

2- Please check whether the program works with the following changes:

ctx->mon_listener = mtcp_socket(ctx->mctx, AF_INET,
				MOS_SOCK_MONITOR_RAW, 0);

icmp_event = mtcp_define_event(MOS_ON_PKT_IN, CatchICMP, NULL);

[VuXuanPhong]

Hello @ajamshed

I hope you are doing well :D. Thank you so much for your reply, I learned a lot. Sorry about my late reply. I read your reply many times to understand.

At the first time read mOS paper, since the flow keyword has many values, I thought FT_DIR is also a multi-event filter function. Now, I understand that mSnort evaluates the sub-rule-options such as flow: to_server or flow:to_client.

I will study on Snort 2.9.x version since having a few documents research on this version. As you suggested, I will do an in-depth study of the two modules: Snort init and rule_parser modules.

I will test the above program according to your instructions.

Thank you so much 👍