itevad – How to write your own protocol and its stack – part 7

Previous_Part_6

From this post, we will have to learn Flex and Bison, which are assumed to have very bizarre grammars to write and understand. Fortunately, we will step forwardly from Flex standalone, Flex+Bison and Flex+Bison+C finally. Please kindly note that all these post would NOT teach you to become the expert of Flex/Bison but show the basic ideas using Flex/Bison to write a stack of a protocol for text decoding, like Itevad. Here we will make sure our Flex/Bison work and try to make a lexer at first using Flex. Let’s go!

/* Install and make sure flex/bison work */
[root@localhost ItevadBin]# which flex
/usr/bin/flex
[root@localhost ItevadBin]# flex -V
flex version 2.5.4
[root@localhost ItevadBin]# which bison
/usr/bin/bison
[root@localhost ItevadBin]# bison -V
bison (GNU Bison) 2.3
Written by Robert Corbett and Richard Stallman.

Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
[root@localhost ItevadBin]#


/* Flex stand alone for scanning Text Itevad Protocol */
[root@localhost ItevadBin]# cat itevadTxtFlexStandAlone.l
/*
Itevad Protocol Text Encoding Lexical Analyzer Stand Alone Example
April 6, 2012
http://daveti.blog.com
Reference: itevad.abnf
NOTE: This is a Flex file NOT Lex!
*/

%{
#include <stdlib.h>
#include <string.h>
#include <stdio.h>

enum yytokentype
{
 DOT,
 LSB,
 RSB,
 EQUAL,
 COLON,
 LBRKT,
 RBRKT,
 SLASH,
 ItevadToken,
 TransToken,
 ReplyToken,
 AskToken,
 AnswerToken,
 quotedString,
 IPv4Address,
 NUMBER
};
%}

/* Configure our lexer */
%option stack noyywrap case-insensitive

%%

"." { return DOT; } /* Dot */
"[" { return LSB; } /* Left square bracket */
"]" { return RSB; } /* Right square bracket */
"=" { return EQUAL; } /* Equal */
":" { return COLON; } /* Colon */
"{" { return LBRKT; } /* Left brace */
"}" { return RBRKT; } /* Right brace */
"/" { return SLASH; } /* Slash */
"Itevad" { return ItevadToken; } /* Itevad Token */
"Transaction" { return TransToken; } /* Transaction Token */
"Reply" { return ReplyToken; } /* Reply Token */
"Ask" { return AskToken; } /* Ask Token */
"Answer" { return AnswerToken; } /* Answer Token */
"(.*)" { printf("Debug - quoted string: %sn", yytext);
 return quotedString;
 } /* Quoted String */
[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3} { /* IPv4 address handling here */
 printf("Debug - IPv4 address: %sn", yytext);
 return IPv4Address;
 } /* IPv4 Address */
[0-9]+ { printf("Debug - number: %sn", yytext);
 return NUMBER;
 } /* Number */
[ trnv] { /* Ignore */ } /* White spaces */

%%

int main( int argc, char **argv)
{
 int token;
 while ( token = yylex())
 {
 printf("Token: %dn", token);
 }

 return 0;
}
[root@localhost ItevadBin]#


/* Generate and run itevadFlexBin */
[root@localhost ItevadBin]# flex itevadTxtFlexStandAlone.l
[root@localhost ItevadBin]# gcc -o itevadFlexBin lex.yy.c -lfl
[root@localhost ItevadBin]# ./itevadFlexBin
Itevad/1 [49.49.49.49]:7777
Transaction = 65535 {
        Ask = "who r you?"
}
Token: 8
Token: 7
Debug - number: 1
Token: 15
Token: 1
Debug - IPv4 address: 49.49.49.49
Token: 14
Token: 2
Token: 4
Debug - number: 7777
Token: 15
Token: 9
Token: 3
Debug - number: 65535
Token: 15
Token: 5
Token: 11
Token: 3
Debug - quoted string: "who r you?"
Token: 13
Token: 6
[root@localhost ItevadBin]#
Posted in Dave's Tools, H.248/MEGACO/EGCP, Programming, Stuff about Compiler | Tagged , , | Leave a comment

itevad – How to write your own protocol and its stack – part 6

Previous_Part_5

From this post, we will start our Flex and Bison journey. Before that, we will need a clarification for BNF and ABNF again and give ABNF description of Iitevad accordingly. As mentioned in General Background, BNF and ABNF are the guidelines to write lexer and parser for text decoding with Flex and Bison. However, there may be a gap between BNF/ABNF description and detailed implementation of lexer/parser, like what happens in Itevad. Our ultimate goal is to make our lexer/parser very close to BNF/ABNF description. Meanwhile, this ultimate goal may increase the complexity of lexer/parser, which may be not so favorable for stack writing. During the stack writing of Itevad, we will find the balance between ABNF and real lexer/parser.

/* ABNF description for Text Itevad Protocol */
[root@localhost ItevadBin]# cat itevad.abnf
;
; Itevad Protocol Text Encoding ABNF Example
; April 6, 2012
; http://daveti.blog.com
;

itevadMessage = LWSP message
message = ItevadToken SLASH Version SEP ip4Address SEP messageBody
messsageBody = (messageRequest / messageReply)
messageRequest = TransToken EQUAL TransactionID LBRKT askContent RBRKT
messageReply = ReplyToken EQUAL TransactionId LBRKT answerContent RBRKT
askContent = AskToken EQUAL quotedString
answerContent = AnswerToken EQUAL quotedString
Version = 1*2(DIGIT) ; "0".."99"
ip4Address = address [":" portNumber]
address = "[" IPv4address "]"
IPv4address = V4hex DOT V4hex DOT V4hex DOT V4hex
V4hex = 1*3(DIGIT) ; "0".."255"
TransactionId = UINT32
portNumber = UINT16

UINT16 = 1*5(DIGIT) ; %x0-FFFF
UINT32 = 1*10(DIGIT) ; %x0-FFFFFFFF
DIGIT = %x30-39 ; 0-9
DOT = "."
DQUOTE = 0x22 ; "
ALPHA = %x41-5A / %x61-7A ; A-Z / a-z
SafeChar = DIGIT / ALPHA / "+" / "-" / "&" / "!" / "_" / "/" / "'" / "?" / "@" /
 "^" / "`" / "~" / "*" / "$" / "" / "(" / ")" / "%" / "|" / "."
RestChar = ";" / "[" / "]" / "{" / "}" / ":" / "," / "#" / "<" / ">" / "="
CR = %x0D ; Carriage return
LF = %x0A ; linefeed
SP = %x20 ; space
HTAB = %x09 ; horizontal tab
COLON = ":" ; colon
WSP = SP / HTAB ; white space
COMMENT = ";" *(SafeChar / RestChar / WSP / %x22) EOL
EOL = (CR [LF] / LF)
LWSP = *(WSP / COMMENT / EOL)
SLASH = "/"
SEP = (WSP / EOL / COMMENT) LWSP
EQUAL = "="
LBRKT = LWSP "{" LWSP
RBRKT = LWSP "}" LWSP
quotedString = DQUOTE *(SafeChar / EOL / %x80-FF / RestChar / WSP) DQUOTE

ItevadToken = "Itevad"
TransToken = "Transaction"
ReplyToken = "Reply"
AskToken = "Ask"
AnswerToken = "Answer"
[root@localhost ItevadBin]#
Posted in Dave's Tools, H.248/MEGACO/EGCP, Programming, Stuff about Compiler | Tagged , , , , | Leave a comment

itevad – How to write your own protocol and its stack – part 5

Previous_Part_4

In this post, we will begin our Itevad Protocol Text Encoder. Again, with the help of ASN.1 compiler, we have already got the structure of Itevad itself. Moreover, we have implemented binary encoder and decoder in previous post. For text encoder, all we need to do is parse the structure of Itevad msg and then translate it into text. Cheers!

/* Message examples for Text ItevadProtocol */
[root@localhost ItevadBin]# cat itevad_text_msg.example
Itevad/1 [31.31.31.31]:7777
Transaction = 1 {
 Ask = "who r you?"
}

Itevad/2 [11.11.11.11]:6666
Reply = 1 {
 Answer = "i am itevad"
}
[root@localhost ItevadBin]#


/* Text encoder for Text Itevad Protocol */
[root@localhost Itevad2]# cat itevadTxtEncoder.c
/*
Itevad Protocol Text Encoder Example
http://daveti.blog.com
April 5, 2012
Reference: Using the Open Source ASN.1 Compiler
by Lev Walkin
*/

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include "ItevadMessage.h"

#define TEXT_ENCODE_BUFFER_SIZE_MAX 1000

#define PTR_MOVE( step, forward, backward) 
 { 
 forward += step; 
 backward -= step; 
 if ( backward < 0) 
 { 
 fprintf( stderr, "No buffer left" 
 " - backward: %dn", backward); 
 } 
 }

/*
Text encoding helper function
to convert OCTET_STRING to string.
*/
char *txt_octetString2String( OCTET_STRING_t *octetString)
{
 /* Defensive checking */
 if ( octetString == NULL)
 {
 fprintf( stderr, "Invalid parametern");
 return NULL;
 }
 if ( octetString->size <= 0)
 {
 fprintf( stderr, "Empty Octet stringn");
 return NULL;
 }

 /* Allocate the string buffer from HEAP */
 char *string = (char *)calloc( 1, (octetString->size+1));

 /* Convert the uint8_t into ASCII characters */
 int i;
 for ( i = 0; i < octetString->size; i++)
 {
 string[ i] = (char)octetString->buf[ i];
 }

 /* Though redundant - just make it clear:) */
 string[ octetString->size] = '';

 return string;
}

/*
Text encoding core function
to convert the ItevadMessage_t type
into Text based Itevad message.
*/
int txt_encode( char *encodeBuffer, ItevadMessage_t *itevadMsg)
{
 /* Defensive checking */
 if ( (encodeBuffer == NULL) || (itevadMsg == NULL))
 {
 fprintf( stderr, "Invalid parametersn");
 return -1;
 }

 /* Go thru itevadMsg to do Text encoding */

 char *txtPtr = encodeBuffer;
 int lengthOfBuffer = TEXT_ENCODE_BUFFER_SIZE_MAX;
 int numOfChar = 0;

 /* Construct msg header */
 numOfChar = snprintf( txtPtr, lengthOfBuffer,
 "Itevad/%d ", itevadMsg->version);
 PTR_MOVE( numOfChar, txtPtr, lengthOfBuffer);

 /* Construct IP address */
 if ( itevadMsg->ip4Address.address.size != 4)
 {
 fprintf( stderr, "Invalid IP address size: %dn",
 itevadMsg->ip4Address.address.size);
 return -1;
 }
 numOfChar = snprintf( txtPtr, lengthOfBuffer,
 "[%u.%u.%u.%u]",
 itevadMsg->ip4Address.address.buf[ 0],
 itevadMsg->ip4Address.address.buf[ 1],
 itevadMsg->ip4Address.address.buf[ 2],
 itevadMsg->ip4Address.address.buf[ 3]);
 PTR_MOVE( numOfChar, txtPtr, lengthOfBuffer);

 /* Construct the port number if it is available */
 if ( itevadMsg->ip4Address.portNumber != NULL)
 {
 numOfChar = snprintf( txtPtr, lengthOfBuffer,
 ":%ld", *(itevadMsg->ip4Address.portNumber));
 PTR_MOVE( numOfChar, txtPtr, lengthOfBuffer);
 }

 /* Move to a new line */
 numOfChar = snprintf( txtPtr, lengthOfBuffer, "%s", "n");
 PTR_MOVE( numOfChar, txtPtr, lengthOfBuffer);

 /* Construct the Transaction */
 long transId = 0;
 char *contentP = NULL;
 switch ( itevadMsg->messageBody.present)
{
 case MessageBody_PR_messageRequest:
 {
 /* Get the transId */
 if ( asn_INTEGER2long(
 &(itevadMsg->messageBody.choice.messageRequest.transactionId),
 &(transId)) == -1)
 {
 fprintf( stderr, "Encode transId in messageRequest errorn");
 return -1;
 }

 /* Get the ask content */
 contentP = txt_octetString2String(
 &(itevadMsg->messageBody.choice.messageRequest.askContent));

 numOfChar = snprintf( txtPtr, lengthOfBuffer,
 "Transaction = %ld {n"
 "tAsk = "%s"n"
 "}n",
 transId, contentP);
 }
 break;

 case MessageBody_PR_messageReply:
 {
 /* Get the transId */
 if ( asn_INTEGER2long(
 &(itevadMsg->messageBody.choice.messageReply.transactionId),
 &(transId)) == -1)
 {
 fprintf( stderr, "Encode transId in messageReply errorn");
 return -1;
 }

 /* Get the answer content */
 contentP = txt_octetString2String(
 &(itevadMsg->messageBody.choice.messageReply.answerContent));

 numOfChar = snprintf( txtPtr, lengthOfBuffer,
 "Reply = %ld {n"
 "tAnswer = "%s"n"
 "}n",
 transId, contentP);
 }
 break;

 default:
 {
 fprintf( stderr, "Invalid message type: %dn",
 itevadMsg->messageBody.present);
 return -1;
 }
 break;
 }

 free( contentP);
 PTR_MOVE( numOfChar, txtPtr, lengthOfBuffer);

 return (txtPtr - encodeBuffer);
}

int main( int ac, char **av)
{
 /* Type to encode */
 ItevadMessage_t *itevadMsg;

 /* Return value for Text encoding */
 int ec;

 /* Text encoding buffer */
 char textEncodeBuffer[ TEXT_ENCODE_BUFFER_SIZE_MAX] = {0};

 /* Allocate the ItevadMessage_t */
 itevadMsg = calloc( 1, sizeof(ItevadMessage_t));
 if ( !itevadMsg)
 {
 perror("calloc() failed");
 exit(71); /* better, EX_OSERR */
 }

 /* Initialize the ItevadMessage members */

 /* Set version number - 1 */
 itevadMsg->version = 1;

 /* Set IPv4 address - "1111" */
 if ( OCTET_STRING_fromString( &(itevadMsg->ip4Address.address), "1111") == -1)
 {
 perror("OCTET_STRING_fromString() failed for encoding 'address'");
 exit(71);
 }

 /* Set port number - 7777 */
 long portNumber = 7777;
 itevadMsg->ip4Address.portNumber = &portNumber;

 /* Set the msg as a request */
 itevadMsg->messageBody.present = MessageBody_PR_messageRequest;

 /* Set the transId - 0xffff */
 long transId = 0xffff;
 if ( asn_long2INTEGER( &(itevadMsg->messageBody.choice.messageRequest.transactionId),
 transId) == -1)
 {
 perror("asn_long2INTEGER() failed for encoding 'transId'");
 exit(71);
 }

 /* Set the askContent - 'who r you?' */
 const char* askContent = "who r you?";
 if ( OCTET_STRING_fromString( &(itevadMsg->messageBody.choice.messageRequest.askContent),
 askContent) == -1)
 {
 perror("OCTET_STRING_fromString() failed for encoding 'askContent'");
 exit(71);
 }

 /* Text encode the data if filename is given */
 if ( ac < 2)
 {
 fprintf( stderr, "Specify filename for Text outputn");
 }
 else
 {
 /* NOTE: need memory deallocaiton before exit ! */

 /* Encode the ItevadMessage type as Text */
 ec = txt_encode( textEncodeBuffer, itevadMsg);

 if ( ec == -1)
 {
 fprintf( stderr, "Could not encode ItevadMessagen");
 exit(65); /* Better, EX_DATAERR */
 }
 else
 {
 /* Write the encoded msg into file */
 const char *filename = av[ 1];
 FILE *fp = fopen( filename, "wb"); /* for Text output */
 if ( !fp)
 {
 perror( filename);
 exit(71);
 }
 fprintf( fp, "%s", textEncodeBuffer);
 fclose( fp);

 fprintf( stderr, "Created %s with Text encoded ItevadMessagen", filename);
 }
 } /* else... */

 /* Also print the constructured Itevad XER encoded (XML) */
 xer_fprint( stdout, &asn_DEF_ItevadMessage, itevadMsg);

 /* NOTE: Need memory deallocaiton here! */

 /* Encoding finished successfully */
 return 0;
}
[root@localhost Itevad2]#


/* Generate and run itevadTxtEncoderBin */
[root@localhost itevad2]# gcc -I. -o itevadTxtEncoderBin *.c
[root@localhost Itevad2]# ./itevadTxtEncoderBin itevad_text.msg
Created itevad_text.msg with Text encoded ItevadMessage
<ItevadMessage>
    <version>1</version>
    <ip4Address>
        <address>31 31 31 31</address>
        <portNumber>7777</portNumber>
    </ip4Address>
    <messageBody>
        <messageRequest>
            <transactionId>65535</transactionId>
            <askContent>who r you?</askContent>
        </messageRequest>
    </messageBody>
</ItevadMessage>
[root@localhost Itevad2]# cat itevad_text.msg
Itevad/1 [49.49.49.49]:7777
Transaction = 65535 {
 Ask = "who r you?"
}
[root@localhost Itevad2]#
Posted in Dave's Tools, H.248/MEGACO/EGCP, Programming, Stuff about Compiler | Tagged , | Leave a comment

itevad – How to write your own protocol and its stack – part 4

Previous_Part_3

In this post, we will continue our Itevad Protocol Binary Ber Decoder. With the help of ANS.1 compiler, not only do we get the structural description of Itevad protocol itself but also the necessary converting functions used to convert between ASN.1 types and C basic data types, as well as the corresponding encoder and decoder function calls per different binary encoding ways, like Per, Ber and Xer. Let’s check the code!

/* Ber decoder for Binary Itevad Protocol */
[root@localhost Itevad2]# cat itevadBerDecoder.c
/*
Itevad Protocol Binary Ber Decoder Example
April 1, 2012
http://daveti.blog.com
Reference: Using the Open Source ASN.1 Compiler
by Lev Walkin
*/

#include <stdio.h>
#include <sys/types.h>
#include "ItevadMessage.h"

int main( int ac, char **av)
{
 char buf[ 1024]; /* Temp buffer */
 ItevadMessage_t *itevadMsg = NULL; /* Type to decode */
 asn_dec_rval_t rval; /* Decoder return value */
 FILE *fp; /* Input file handler */
 size_t size; /* Number of bytes read */
 char *filename; /* Input file name */

 /* Require a single filename argument */
 if ( ac != 2)
 {
 fprintf( stderr, "Usage: %s <file.ber>n", av[ 0]);
 exit(64); /* better, EX_USAGE */
 }
 else
 {
 filename = av[ 1];
 }

 /* Open input file as read-only binary */
 fp = fopen( filename, "rb");
 if ( !fp)
 {
 perror( filename);
 exit(66); /* better, EX_NOINPUT */
 }

 /* Read up to the buffer size */
 size = fread( buf, 1, sizeof( buf), fp);
 fclose( fp);
 if ( !size)
 {
 fprintf( stderr, "%s: Empty or brokenn", filename);
 exit(65); /* better, EX_DATAERR */
 }

 /* Decode the input buffer as Itevad type */
 rval = ber_decode( 0, &asn_DEF_ItevadMessage, (void **)&itevadMsg, buf, size);
 if ( rval.code != RC_OK)
 {
 fprintf( stderr, "%s: Broken ItevadMessage decoding at byte %ldn",
 filename, (long)rval.consumed);
 exit(65); /* better, EX_DATAERR */
 }

 /* Print the decoded Itevad type as XML */
 xer_fprint( stdout, &asn_DEF_ItevadMessage, itevadMsg);

 /* Decoding finished successfully */
 return 0;
}
[root@localhost itevad2]#


/* Generate and run itevadBerDecoderBin */
[root@localhost itevad2]# gcc -I. -o itevadBerDecoderBin *.c
[root@localhost Itevad2]# ./itevadBerDecoderBin itevad_binary.msg
<ItevadMessage>
    <version>1</version>
    <ip4Address>
        <address>31 31 31 31</address>
        <portNumber>7777</portNumber>
    </ip4Address>
    <messageBody>
        <messageRequest>
            <transactionId>65535</transactionId>
            <askContent>who r you?</askContent>
        </messageRequest>
    </messageBody>
</ItevadMessage>
[root@localhost itevad2]#
Posted in Dave's Tools, H.248/MEGACO/EGCP, Programming, Stuff about Compiler | Tagged , , | Leave a comment

itevad – How to write your own protocol and its stack – part 3

Previous_Part_2

In this post, we will focus on writing Itevad Protocol Binary Ber Encoder. With the necessary headers and sources generated in Part_2, now we could write our Ber Encoder. Then we will run our encoder and use hexdump to see what is our Itevad msg looking like. Fire!

/* Ber encoder for Binary Itevad Protocol */
[root@localhost itevad2]# cat itevadBerEncoder.c
/*
Itevad Protocol Binary Ber Encoder Example
http://daveti.blog.com
April 1, 2012
Reference: Using the Open Source ASN.1 Compiler
by Lev Walkin
*/

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include "ItevadMessage.h"

/*
 * This is a custom function which writes the
 * encoded output into some FILE stream.
 */
static int
write_out( const void *buffer, size_t size, void *app_key)
{
 FILE *out_fp = app_key;
 size_t wrote;
 wrote = fwrite( buffer, 1, size, out_fp);
 return ((wrote == size) ? 0 : -1);
}

int main( int ac, char **av)
{
 /* Type to encode */
 ItevadMessage_t *itevadMsg;

 /* Encoder return value */
 asn_enc_rval_t ec;

 /* Allocate the ItevadMessage_t */
 itevadMsg = calloc( 1, sizeof(ItevadMessage_t));
 if ( !itevadMsg)
 {
 perror("calloc() failed");
 exit(71); /* better, EX_OSERR */
 }

 /* Initialize the ItevadMessage members */

 /* Set version number - 1 */
 itevadMsg->version = 1;

 /* Set IPv4 address - "1111" */
 if ( OCTET_STRING_fromString( &(itevadMsg->ip4Address.address), "1111") == -1)
 {
 perror("OCTET_STRING_fromString() failed for encoding 'address'");
 exit(71);
 }

 /* Set port number - 7777 */
 long portNumber = 7777;
 itevadMsg->ip4Address.portNumber = &portNumber;

 /* Set the msg as a request */
 itevadMsg->messageBody.present = MessageBody_PR_messageRequest;

 /* Set the transId - 0xffff */
 long transId = 0xffff;
 if ( asn_long2INTEGER( &(itevadMsg->messageBody.choice.messageRequest.transactionId),
 transId) == -1)
 {
 perror("asn_long2INTEGER() failed for encoding 'transId'");
 exit(71);
 }

 /* Set the askContent - 'who r you?' */
 const char* askContent = "who r you?";
 if ( OCTET_STRING_fromString( &(itevadMsg->messageBody.choice.messageRequest.askContent),
 askContent) == -1)
 {
 perror("OCTET_STRING_fromString() failed for encoding 'askContent'");
 exit(71);
 }

 /* BER encode the data if filename is given */
 if ( ac < 2)
 {
 fprintf( stderr, "Specify filename for BER outputn");
 }
 else
 {
 /* NOTE: need memory deallocaiton before exit ! */

 const char *filename = av[ 1];
 FILE *fp = fopen( filename, "wb"); /* for BER output */
 if ( !fp)
 {
 perror( filename);
 exit(71);
 }

 /* Encode the ItevadMessage type as BER (DER) */
 ec = der_encode( &asn_DEF_ItevadMessage, itevadMsg, write_out, fp);
 fclose( fp);

 if ( ec.encoded == -1)
 {
 fprintf( stderr, "Could not encode ItevadMessage (at %s)n",
 ec.failed_type ? ec.failed_type->name : "unknown");
 exit(65); /* Better, EX_DATAERR */
 }
 else
 {
 fprintf( stderr, "Created %s with BER encoded ItevadMessagen", filename);
 }
 } /* else... */

 /* Also print the constructured Itevad XER encoded (XML) */
 xer_fprint( stdout, &asn_DEF_ItevadMessage, itevadMsg);

 /* NOTE: Need memory deallocaiton here! */

 /* Encoding finished successfully */
 return 0;
}
[root@localhost itevad2]#


/* Generate and run itevadBerEncoderBin */
[root@localhost itevad2]# gcc -I. -o itevadBerEncoderBin *.c
[root@localhost Itevad2]# ./itevadBerEncoderBin itevad_binary.msg
Created itevad_binary.msg with BER encoded ItevadMessage
<ItevadMessage>
    <version>1</version>
    <ip4Address>
        <address>31 31 31 31</address>
        <portNumber>7777</portNumber>
    </ip4Address>
    <messageBody>
        <messageRequest>
            <transactionId>65535</transactionId>
            <askContent>who r you?</askContent>
        </messageRequest>
    </messageBody>
</ItevadMessage>
[root@localhost Itevad2]# hexdump -C itevad_binary.msg
00000000  30 24 80 01 01 a1 0a 80  04 31 31 31 31 81 02 1e  |0$.......1111...|
00000010  61 a2 13 a0 11 80 03 00  ff ff 81 0a 77 68 6f 20  |a...........who |
00000020  72 20 79 6f 75 3f                                 |r you?|
00000026
[root@localhost Itevad2]# hexdump -c itevad_binary.msg
0000000   0   $ 200 001 001 241  n 200 004   1   1   1   1 201 002 036
0000010   a 242 023 240 021 200 003   377 377 201  n   w   h   o
0000020   r       y   o   u   ?
0000026
[root@localhost itevad2]#
Posted in Dave's Tools, H.248/MEGACO/EGCP, Programming, Stuff about Compiler | Tagged , , , | 1 Comment

itevad – How to write your own protocol and its stack – part 2

Previous_Part_1

In this part, we will start to play with ASN.1 and asn1c – installing open source ASN.1 compiler, asn1c, writing ANS.1 description of Itevad and then compiling. Please note we will not cover detailed grammar of ASN.1 or how to install asn1c into your machine. All these info could be found from Lev’s website: http://lionet.info/asn1c/blog/ Here we go:)

/* Install and make sure ASN.1 compiler work */
[root@localhost ItevadBin]# which asn1c
/usr/local/bin/asn1c
[root@localhost ItevadBin]# asn1c -h
ASN.1 Compiler, v0.9.21
Copyright (c) 2003, 2004, 2005, 2006 Lev Walkin <vlm@lionet.info>
Usage: asn1c [options] file ...
Options:
  -E                    Run only the ASN.1 parser and print out the tree
  -F                    During -E operation, also perform tree fixing

  -P                    Concatenate and print the compiled text
  -R                    Restrict output (tables only, no support code)
  -S <dir>              Directory with support (skeleton?) files
                        (Default is "/usr/local/share/asn1c")
  -X                    Generate and print the XML DTD

  -Werror               Treat warnings as errors; abort if any warning
  -Wdebug-lexer         Enable verbose debugging output from lexer
  -Wdebug-fixer         --//-- semantics processor
  -Wdebug-compiler      --//-- compiler

  -fbless-SIZE          Allow SIZE() constraint for INTEGER etc (non-std.)
  -fcompound-names      Disambiguate C's struct NAME's inside top-level types
  -findirect-choice     Compile members of CHOICE as indirect pointers
  -fknown-extern-type=<name>    Pretend the specified type is known
  -fnative-types        Use "long" instead of INTEGER_t whenever possible, etc.
  -fno-constraints      Do not generate constraint checking code
  -fno-include-deps     Do not generate courtesy #includes for dependencies
  -funnamed-unions      Enable unnamed unions in structures
  -fskeletons-copy      Force copying the support files

  -gen-PER              Generate PER support code
  -pdu=auto             Generate PDU table (discover PDUs automatically)

  -print-class-matrix   Print out the collected object class matrix (debug)
  -print-constraints    Explain subtype constraints (debug)
  -print-lines          Generate "-- #line" comments in -E output
[root@localhost ItevadBin]#


/* ASN.1 description for Itevad Protocol */
[root@localhost itevad2]# cat itevad.asn1
/*
 * Itevad - an example protocol
 * which is used to do nothing or nothing
 * Name: itevad.asn1
 * dave.tian@alcatel-lucent.com
 * Jan 6, 2011
 * http://daveti.blog.com
 */

ITEVAD DEFINITIONS AUTOMATIC TAGS ::=
BEGIN

ItevadMessage ::= SEQUENCE
{
 version Version,
 ip4Address IP4Address,
 messageBody MessageBody
}

Version ::= INTEGER(0..99)

IP4Address ::= SEQUENCE
{
 address OCTET STRING (SIZE(4)),
 portNumber INTEGER(0..65536) OPTIONAL
}

MessageBody ::= CHOICE
{
 messageRequest MessageRequest,
 messageReply MessageReply
}

MessageRequest ::= SEQUENCE
{
 transactionId TransactionId,
 askContent AskContent
}

MessageReply ::= SEQUENCE
{
 transactionId TransactionId,
 answerContent AnswerContent
}

TransactionId ::= INTEGER(0..4294967295) -- 32-bit unsigned integer, 0xffffffff

AskContent ::= IA5String(SIZE(1..1000))

AnswerContent ::= IA5String(SIZE(1..1000))

END
[root@localhost itevad2]#


/* Generate structure headers and sources for encoding/decoding Binary Itevad Protocol */
[root@localhost itevad2]# asn1c -fnative-types itevad.asn1
Compiled ItevadMessage.c
Compiled ItevadMessage.h
Compiled Version.c
Compiled Version.h
Compiled IP4Address.c
Compiled IP4Address.h
Compiled MessageBody.c
Compiled MessageBody.h
Compiled MessageRequest.c
Compiled MessageRequest.h
Compiled MessageReply.c
Compiled MessageReply.h
Compiled TransactionId.c
Compiled TransactionId.h
Compiled AskContent.c
Compiled AskContent.h
Compiled AnswerContent.c
Compiled AnswerContent.h
Symlinked /usr/local/share/asn1c/INTEGER.h      -> INTEGER.h
Symlinked /usr/local/share/asn1c/NativeEnumerated.h     -> NativeEnumerated.h
Symlinked /usr/local/share/asn1c/IA5String.h    -> IA5String.h
Symlinked /usr/local/share/asn1c/IA5String.c    -> IA5String.c
Symlinked /usr/local/share/asn1c/INTEGER.c      -> INTEGER.c
Symlinked /usr/local/share/asn1c/NativeEnumerated.c     -> NativeEnumerated.c
Symlinked /usr/local/share/asn1c/NativeInteger.h        -> NativeInteger.h
Symlinked /usr/local/share/asn1c/NativeInteger.c        -> NativeInteger.c
Symlinked /usr/local/share/asn1c/constr_CHOICE.h        -> constr_CHOICE.h
Symlinked /usr/local/share/asn1c/constr_CHOICE.c        -> constr_CHOICE.c
Symlinked /usr/local/share/asn1c/constr_SEQUENCE.h      -> constr_SEQUENCE.h
Symlinked /usr/local/share/asn1c/constr_SEQUENCE.c      -> constr_SEQUENCE.c
Symlinked /usr/local/share/asn1c/asn_application.h      -> asn_application.h
Symlinked /usr/local/share/asn1c/asn_system.h   -> asn_system.h
Symlinked /usr/local/share/asn1c/asn_codecs.h   -> asn_codecs.h
Symlinked /usr/local/share/asn1c/asn_internal.h -> asn_internal.h
Symlinked /usr/local/share/asn1c/OCTET_STRING.h -> OCTET_STRING.h
Symlinked /usr/local/share/asn1c/OCTET_STRING.c -> OCTET_STRING.c
Symlinked /usr/local/share/asn1c/BIT_STRING.h   -> BIT_STRING.h
Symlinked /usr/local/share/asn1c/BIT_STRING.c   -> BIT_STRING.c
Symlinked /usr/local/share/asn1c/asn_codecs_prim.c      -> asn_codecs_prim.c
Symlinked /usr/local/share/asn1c/asn_codecs_prim.h      -> asn_codecs_prim.h
Symlinked /usr/local/share/asn1c/ber_tlv_length.h       -> ber_tlv_length.h
Symlinked /usr/local/share/asn1c/ber_tlv_length.c       -> ber_tlv_length.c
Symlinked /usr/local/share/asn1c/ber_tlv_tag.h  -> ber_tlv_tag.h
Symlinked /usr/local/share/asn1c/ber_tlv_tag.c  -> ber_tlv_tag.c
Symlinked /usr/local/share/asn1c/ber_decoder.h  -> ber_decoder.h
Symlinked /usr/local/share/asn1c/ber_decoder.c  -> ber_decoder.c
Symlinked /usr/local/share/asn1c/der_encoder.h  -> der_encoder.h
Symlinked /usr/local/share/asn1c/der_encoder.c  -> der_encoder.c
Symlinked /usr/local/share/asn1c/constr_TYPE.h  -> constr_TYPE.h
Symlinked /usr/local/share/asn1c/constr_TYPE.c  -> constr_TYPE.c
Symlinked /usr/local/share/asn1c/constraints.h  -> constraints.h
Symlinked /usr/local/share/asn1c/constraints.c  -> constraints.c
Symlinked /usr/local/share/asn1c/xer_support.h  -> xer_support.h
Symlinked /usr/local/share/asn1c/xer_support.c  -> xer_support.c
Symlinked /usr/local/share/asn1c/xer_decoder.h  -> xer_decoder.h
Symlinked /usr/local/share/asn1c/xer_decoder.c  -> xer_decoder.c
Symlinked /usr/local/share/asn1c/xer_encoder.h  -> xer_encoder.h
Symlinked /usr/local/share/asn1c/xer_encoder.c  -> xer_encoder.c
Symlinked /usr/local/share/asn1c/per_support.h  -> per_support.h
Symlinked /usr/local/share/asn1c/per_support.c  -> per_support.c
Symlinked /usr/local/share/asn1c/per_decoder.h  -> per_decoder.h
Symlinked /usr/local/share/asn1c/per_decoder.c  -> per_decoder.c
Symlinked /usr/local/share/asn1c/per_encoder.h  -> per_encoder.h
Symlinked /usr/local/share/asn1c/per_encoder.c  -> per_encoder.c
Symlinked /usr/local/share/asn1c/converter-sample.c     -> converter-sample.c
Generated Makefile.am.sample
[root@localhost itevad2]# rm converter-sample.c
rm: remove symbolic link `converter-sample.c'? yes
Posted in Dave's Tools, H.248/MEGACO/EGCP, Programming, Stuff about Compiler | Tagged , , | Leave a comment

itevad – How to write your own protocol and its stack – part 1

From this post, as well as the following 4~6 posts, I will try to illustrate how to write your own protocol and its stack, step by step. You will learn all the related stuffs to write a carrier grade telecommunication protocol called ‘Itevad’ supporting both binary and text encoding/decoding. All the code could be found in repository of github called ‘itevad’. Have fun:)
Project Name: itevad
Destination: Itevad Protocol Design and Stack Implementation
Language: ASN.1, BNF, ABNF, Flex, Bison, C
Project Web: http://github.com/daveti/itevad
Git Read Only: https://github.com/daveti/itevad.git
Part 1 – General background

1. Carrier grade telecommunication protocol
A carrier grade telecommunication protocol usually supports both binary and text encoding/decoding with high performance. Binary format is hard to read manually but saving network traffic, while text format has the other side of the story. For binary format, performance mostly relies on TLV (tag, length, value) analysis. However, to achieve high performance, text format needs the help of flex and bison, especially when the protocol itself is complicated and even may be impossible to do pure string parsing.
2. ASN.1 and asn1c
For binary format, if we do not care about the detailed mapping between different tags and possible length and value, which means we focus on translating the protocol itself into binary format rather than encoding and decoding, then you have to know ASN.1. ASN.1 is a standard language used to describe the protocol with structure style.  In the following post, we will see how Itevad protocol looks like when it is written in ANS.1. Once we have the ASN.1 description, we could use any ANS.1 compiler to generate all the source code needed to encode and decode binary formatted protocol. asn1c is an open source ANS.1 compiler written by Lev Walkin and also the one we are using for Itevad.
NOTE: For detailed info about ASN.1 and ans1c, please refer to the link category for ‘H.248’ and ‘Compiler’ on your right side. ‘EGCP’ (a H.248 like protocol from Genband) may be the case where ANS.1 could not help.
3. BNF and ABNF
BNF is a metalanguage used to describe the grammar of certain language. As you could imagine, BNF is used to describe programming languages and telecommunication protocols. ABNF is an Augmented BNF with its own syntax and rules. Communication protocols from IETF and ITU-T are mostly have ABNF description if they support text format, like H.248.1. In my opinion, BNF/ABNF gives the guideline on how to write a flex/bison stack for text decoding.
4. Flex
As its ancestor lex, Flex is a fast lexer used to do lexical analysis, which is usually the front-end part of a compiler. The main job for Flex is to scan the text context and recognize certain pattern as a ‘token’. That is why Flex is also called tokenizer sometime. All detailed Flex info could be found under stuff of ‘Compiler’ at your right side.
5. Bison
As its ancestor yacc, Bison is a parser used to do syntax analysis, whose input is usually from output of Flex in a compiler system. The main job for Bison is to scan the text context to determine ‘meaning’ of the whole passage with the input of each ‘token’ from Flex. All detailed Bison info could be found under stuff of ‘Compiler’ at your right side.
6. Itevad
Itevad Protocol, which we will design and implement its stack step by step later, is a carrier grade telecommunication protocol supporting both binary and text encoding/decoding. As kind of difficult to show the binary msg of this protocol, here lists the text msg examples for a general view:
/* Message examples for Text ItevadProtocol */
[root@localhost ItevadBin]# cat itevad_text_msg.example
Itevad/1 [31.31.31.31]:7777
Transaction = 1 {
 Ask = "who r you?"
}

Itevad/2 [11.11.11.11]:6666
Reply = 1 {
 Answer = "i am itevad"
}
[root@localhost ItevadBin]#
Posted in Dave's Tools, H.248/MEGACO/EGCP, Programming, Stuff about Compiler | Tagged , , , , , , , , | Leave a comment

jmgsim again – jmgsim-ov, git and github fantastic!

While Kenny (kenny.du@alcatel-lucent.com) is refactoring ‘jmgsim’ in java.net, I am looking for a better code repository to hold my initial, tested and verified source code of jmgsim for my personal reference as well as design mapping in first_post_of_jmgsim. Finally I have found my ultimate code repository – github!

Project Name: jmgsim-ov
Destination: A Java implementation of H.248 Media Gateway Simulator - original version
Language: Java
IDE: NetBeans 7.1
Project Web: http://github.com/daveti/jmgsim
Git Read Only: https://github.com/daveti/jmgsim.git
Why github?
1. kenai is good but closing inprog...
2. java.net is good but restricted with Java related code.
3. java.net needs community manager's approvement to make the project public.
4. Google Code is good but I just do not like the web style...
5. github is NOT restricted with any kind of code!
6. All projects in github could be public without approvement!
7. git is similar with svn!
8. git/github rookie guide is so friendly!
Posted in Dave's Tools, Programming | Tagged , , , , , , | Leave a comment

uninit member in destructor – Coverity UNINIT and UNINIT_CTOR checkers

Recently we encountered a serious bug like the code below – destructor function delete one member ptr, which was never init’d. At first, we were assuming checker UNINIT could help figure it out as we do not see any difference between destructor and other common functions. Unfortunately, we failed. Then we tried to turn on checker UNINIT_CTOR, which is used to detect the uninit non static members within the constructor. On one hand, it indeed gave the warning like it is expected; on the other hand, this checker did not care about other functions except constructor, which may cause false alarm like below. No matter the golden rule ‘member init within constructor’ is followed or not, the key point would be if this uninit issue could be reported within desctructor – currently seems not.

28      /* daveti - cov code starts */
29      class Daveti
30      {
31      public:
32              int a;
33              int b;
34      };
35
36      class MyCov
37      {
38      public:
39              int c;
/*
Event member_decl: Class member declaration for ""myPtr"".
Also see events: [uninit_member]
*/
40              Daveti *myPtr;
41
42              MyCov()
43              {
44                      c = 0xffff;
/*
Event uninit_member: Non-static class member ""myPtr"" is not initialized in this constructor nor in any functions that it calls.
Also see events: [member_decl]
*/
45              }
46
47              ~MyCov()
48              {
49                      delete(myPtr);
50              }
51
52              void myPrintf()
53              {
54                      printf("c = %d, Daveti.a = %dn", c, myPtr->a);
55              }
56      };
57
58      void myFunc(MyCov *ptr)
59      {
60              // Init the wild ptr
61              ptr->myPtr = new Daveti();
62      }
63      /* daveti - cov code ends */
64
65      void
66      VPEmain(ULONG dummy0, ULONG init_type)
67      {
68      /* daveti - cov code starts */
69      MyCov *myCovObj = new MyCov();
70      myFunc(myCovObj); /* daveti - member init outside constructor */
71      myCovObj->myPrintf();
72      delete(myCovObj); /* daveti - issued line if line 70 is commented */
73      /* daveti - cov code ends */
Posted in Programming, Static Code Analysis | Tagged , | Leave a comment

jmgsim – a Java implementation of H.248 media gateway simulator

‘jmgsim’ is a Java implementation of H.248 Media Gateway (MG) simulator and Media Gateway Controller (MGC) simulator used for testing real MG/MGC. It is coded within NetBeans 7.1 and has been published as a public project at ‘java.net’, as ‘kenai’ is no long accepting new project and ‘java.net’ is taking the place of ‘kenai’. I am not sure if this post would be an ‘official’ design document for ‘jmgsim’ – I will have a try as I do not think I will write such a doc:) BTW, special thanks to David Lin, who shared his IUA_simulator project for my reference.

Project Name: jmgsim
Destination: A Java implementation of H.248 Media Gateway Simulator
Language: Java
IDE: NetBeans 7.1
Project Web: http://java.net/projects/jmgsim
SVN address: https://svn.java.net/svn/jmgsim~jmgsim
Source Code: http://java.net/projects/jmgsim/sources/jmgsim/show

Design description:
A. Thread:
1. main - taking responsibility of all objects creation and Cli mode
2. CommonWorker - msg recv, decode, process and send
3. MgReg - focusing on sending registration to MGC
4. MgHb - focusing on sending HB to MGC
5. MgcHb - focusing on sending HB to MG

B. Class:
1. Transport - UDP socket, recv and send
2. GlobalConfig - global config data from 'config.properties'
3. MyLogger - logging
4. GwInfo - abstraction for gateway
5. GwInfoDB - management for all GwInfos
6. Cli - command line interface
7. MsgToken - H.248 tokens
8. MsgProc - H.248 msg parsing
9. MsgBuild - H.248 msg construction
10. H248Msg - abstraction for H.248 msg
11. H248MsgDecoded - abstraction for decoding H.248 msg
12. H248Transaction - abstraction of H.248 Transaction
13. H248Action - abstraction of H.248 Action
14. H248Command - abstraction of H.248 Command

Below is a not-so-clear arch pic of 'jmgsim'...
Posted in Dave's Tools, H.248/MEGACO/EGCP, Programming | Tagged , , , , , , | Leave a comment