ANML Programmers Reference

Introduction

This section describes the programming interface for the ANML portion of the Automata Processor SDK. Please refer to the ANML Users Guide for a full description of the ANML Language. The ANML API allows the programmer to programmatically construct automata with a combination of directly adding and connecting elements and/or reading in one or more of the various types of ANML files. Using ANML allows automata to be created in a much more direct manner than using a high-level language such as PCRE.

The ANML Object

An automaton is constructed in an ANML "object." The object must first be created using AP_CreateAnml() which returns a reference used by other ANML APIs to either directly create and connect elements or read in the elements and connections described in external ANML files. Any combination of direct creation or reading external ANML files is possible using the ANML object. Once the automaton has been created, the ANML object is compiled with AP_CompileAnml() to create a loadable automton file. The compliation can also produces an ANML Element Map. Once the object has been used to create the automaton, it can be removed and have all internal resources freed up with AP_DestroyAnml().

The much more direct AP_CompileAnmlFiles() allows a set of one or more ANML files to be loaded then immediately compiled into an automaton, without the need to manage an ANML object. The default path to the current definition of the ANML language, i.e. the ANML "schema," can be found by calling AP_GetAnmlSchemaPath().

ANML Namespaces

The ANML object is the context in which automata are constructed, and many automata can be constructed in the same object. This is done through the use of sub-contexts called "namespaces." A namespace is where AP elements, i.e. STEs, counters, booleans, and, macro references, are created and connected together. There are currently two namespaces in which to construct automata, an "automata network" and a "macro definition." Within namespaces, identifiers of elements must be unique. As described above, automata can be created by loading one or more ANML files with AP_LoadAnmlObjects(), and the type of a namespace can be queried with AP_GetAnmlObjectType(). Equally, elements and connections can be created directly inside namespaces with AP_AddAnmlElement(), AP_AddAnmlEdge(), and AP_AddAnmlEdgeEx(), and the elements that have been added to networks can be managed with AP_GetAnmlIdFromObject(), and AP_GetAnmlObjectFromId(). Once a namespace is created, it can be exported to an ANML file using AP_ExportAnml() before the ANML object is compiled (see below). It's important to note that a namespace object must first be created within the ANML object before elements can be created or used within them. The following sections describe this in more detail.

A note about AP_LoadAnmlObjects(). This API specifies that a generic namespace reference, ap_anml_namespace_t, value will be returned once the ANML file is loaded. In fact, either an ANML network reference, ap_anml_network_t, or a macro definition reference, ap_macro_def_t, can be specified. The point is that a generic reference is interchangeable with the others, since the user typically knows what type of ANML file, either automata network or macro definition, is being loaded.

Automata Networks

An automata network describes the structure of an automaton. It can be created by loading an ANML file with AP_LoadAnmlObjects() which will create the automata network within the ANML object as a side effect, or it can be created directly with AP_CreateAutomataNetwork(), then elements can be created and connected within it as described above.

This tutorial describes How to create a simple Automata Network.

Macro Definitions

A macro definition describes the structure of an automaton that be used over and over. In other words, it is a "blueprint" for an automaton. Macro definitions facilitate modularity and code reuse which allows user-defined functionality to be accumulated into libraries (as described below). A macro definition can be created by loading a macro definition file with AP_LoadAnmlMacro(), which will create the macro definition within an ANML object as a side effect, or it can be created directly with AP_CreateMacroDef(), then elements can be created and connected within it as described above.

This tutorial describes How to create and use a simple macro.

Macro definitions can contain elements that are specific only to macro definitions which can be created programmatically with the following APIs.

Macro ports are created and managed with AP_AddMacroPort(), AP_GetMacroPortFromId(), AP_SetMacroPortInDegree(), and AP_AttachElementToPort().

These tutorials describe Adding ports to macros and Adding report aliases to macros.

Macro parameters can be created with AP_AddMacroParam() and AP_AddMacroParamMulti(). A macro parameter object can be obtained with AP_GetMacroParamFromName(). A parameter substitution is performed with GetMacroParamSubstitutionHolder() and SetMacroParamSubstitution().

This tutorial describes Adding parameters to macros.

The position information for the display of macro definitions in the Workbench can be set with SetMacroLayoutSize() and SetMacroPortLayoutPosition().

Pre-Compilation of Macro Definitions

For each macro that is instantiated with a macro reference, the compiler will ultimately create a complete netlist of AP resources, as described by the macro definition. If there are many references, a lot of the same work is performed over and over. For this reason, the ANML API provides a way to "pre-compile" a macro definition. That is, the resulting netlist can be created "out of context" and stored in a file. Later, when it is to be instantiated in a macro reference, the reference specifies the pre-compiled file. It doesn't matter whether the macro was originally created by loading a macro definition ANML file or built up programmatically. Once a macro definition object has been created, it can be precompiled and stored away. As well, precompiled macro ANML files can be loaded into macro definition objects. Macros are precompiled by using AP_SetMacroDefToBeCompiled() and AP_CompileMacros().

The command-line compiler can be used to precompile ANML macros that can subsequently be referenced in another automata network or macro definition. The following shows how to use apcompile to do so.

apcompile -A -f <resulting pre-compiled macro>.xml <input macro definition file>.xml

ANML Library

Automata networks and macro definitions can reference macro definitions, but the name of the macro definition must be unique. This rule can easily cause name conflicts. Imagine two different set of macros created in two different places that accidentally use the same macro definition name without even knowing it. It's very possible. The way to get around this situation is to use an ANML library. The ANML library is a container object for macro definitions. But what it also does is provide a namespace for macro definitions. For instance, if a macro definition named "foo" were in a library named "bar," if the that macro library were included in an automata network or even another macro definition, you would reference it with the ID, "bar.foo" which makes the macro definition's name unique. This, of course means that library name must be unique, but that is a common situation in almost any programming language or environment.

An ANML library can be created by loading an anml-library file with AP_LoadAnmlLibrary(), which will create the macro library within the ANML object as a side effect, or it can be created directly with AP_CreateAnmlLibrary(). Unlike namespaces, macro definition files must be loaded into ANML libraries using AP_LoadAnmlMacroInLibrary() and macro definitions must be directly created in libraries with AP_CreateMacroDefInLibrary(). Once created, ANML libraries can be exported to an external file with AP_ExportAnmlLibrary(), but only before the containing ANML object is compiled.

The following code snippets create an ANML library that contains a single macro definition.

1 <anml-library>
2  <library-definition id="lib1">
3  <macro-definition id="m1">
4  <header>
5  <interface-declarations>
6  <port id="p0" type="in"/>
7  <port id="p1" type="out"/>
8  </interface-declarations>
9  </header>
10  <body>
11  <port-definitions>
12  <port-in id="p0">
13  <activate-on-event element="ste1"/>
14  </port-in>
15  <port-out id="p1">
16  <activate-from-event element="ste2"/>
17  </port-out>
18  </port-definitions>
19  <state-transition-element id="ste1" symbol-set="b">
20  <activate-on-match element="ste2"/>
21  </state-transition-element>
22  <state-transition-element id="ste2" symbol-set="c"/>
23  </body>
24  </macro-definition>
25  </library-definition>
26 </anml>
#include <micron/ap/ap_anml.h>
ap_anml_library_t CreateAnmlLibrary(ap_anml_t anml)
{
ap_anml_library_t anmlLibrary;
ap_macro_def_t macroDef;
struct ap_anml_element element;
ap_anml_element_ref_t element_b, element_c;
ap_port_ref_t p0, p1;
// initialize the element structure
memset(&element, 0, sizeof(element));
// create an ANML library in the anml object
AP_CreateAnmlLibrary(anml, &anmlLibrary, "lib1");
// create the macro definition in the ANML library
AP_CreateMacroDefInLibrary(anmlLibrary, &macroDef, "m1");
// create the elements that will match "bc" in the macro definition namespace
element.res_type = RT_STE;
element.start = NO_START;
element.symbols = "b";
element.match = 0;
AP_AddAnmlElement(macroDef, &element_b, &element);
element.res_type = RT_STE;
element.start = NO_START;
element.symbols = "c";
element.match = 0;
AP_AddAnmlElement(macroDef, &element_c, &element);
AP_AddAnmlEdge(macroDef, element_b, element_c, 0);
// add an input and output port to the macro definition namespace
AP_AddMacroPort(macroDef, &p0, "p0", PORT_IN);
AP_AddMacroPort(macroDef, &p1, "p1", PORT_OUT);
// attach the input port to the STE that matches "b" and the STE that
// matches "c" to the output port
AP_AttachElementToPort(macroDef, p0, element_b, 0);
AP_AttachElementToPort(macroDef, p1, element_c, 0);
// return the ANML library
return anmlLibrary;
}
1 from micronap.sdk import *
2 
3 def CreateAnmlLibrary(anml):
4 
5  # create an ANML library in the anml object
6  anmlLibrary = anml.CreateAnmlLibrary("lib1")
7 
8  # create the macro definition namespace in the ANML library
9  macroDef = anmlLibrary.CreateMacroDefInLibrary("m1")
10 
11  # create the elements that will match "bc" in the macro definition namespace
12  element_b = macroDef.AddSTE('b')
13  element_c = macroDef.AddSTE('c')
14  macroDef.AddAnmlEdge(element_b, element_c, 0)
15 
16  # add an input and output port to the macro definition namespace
17  p0 = macroDef.AddMacroPort('p0', AnmlDefs.PORT_IN)
18  p1 = macroDef.AddMacroPort('p1', AnmlDefs.PORT_OUT)
19 
20  # attach the input port to the STE that matches "a" and the STE that
21  # matches "c" to the output port
22  macroDef.AttachElementToPort(p0, element_b, 0)
23  macroDef.AttachElementToPort(p1, element_c, 0)
24 
25  # return the ANML library
26  return anmlLibrary
import com.micron.ap.*;
public class CreateAnmlLibraryExample {
public static AnmlLibrary CreateAnmlLibrary(Anml anml) throws ApException {
AnmlLibrary anmlLibrary;
MacroDef macroDef;
ElementRef element_b, element_c;
int p0, p1;
// create the ANML library in the ANML object
anmlLibrary = anml.createAnmlLibrary("lib1");
// create the macro definition in the ANML library
macroDef = anmlLibrary.createMacroDefInLibrary("m1");
// create the elements that will match "bc" in the macro definition namespace
element_b = macroDef.addSTE("b");
element_c = macroDef.addSTE("c");
macroDef.addAnmlEdge(element_b, element_c, 0);
// add an input and output port to the macro definition namespace
p0 = macroDef.addMacroPort("p0", AnmlDefs.PORT_IN);
p1 = macroDef.addMacroPort("p1", AnmlDefs.PORT_OUT);
// attach the input port to the STE that matches "b" and the STE that
// matches "c" to the output port
macroDef.attachElementToPort(p0, element_b, 0);
macroDef.attachElementToPort(p1, element_c, 0);
// return the ANML library
return anmlLibrary;
}
}

ANML Source Files

There are three types of ANML source files. Each is an XML file with a different top-level tag. An ANML source file with the top-level tag, "macro-definition," can contain a single macro definition. An ANML source file with the top-level tag, "anml-library," can contain any number of macro definitions. An ANML source file with the top-level tag, "anml," can contain any number of namespaces, e.g. automata networks and macro definitions. Both anml and anml-library files can include external macro definitions by using the "include-macro" and "include-library" tags. Once an external macro definition is included, the macros contained can be directly referenced. If the macro definition is contained in an ANML library, the library's ID must be used to qualify the macro's ID.

Automaton Construction

Automatona can be directly constructed either programmatically or by authoring an ANML XML. The AP Workbench is a visual authoring tool that can be used for this purpose. The workbench also allows for simulation of Automata and compilation in an IDE.

The following example shows how to create an automata network using the API's and the XML equivalent. When authoring ANML XML outside the workbench the ANML source requires compilation using either the apcompile tool, or programmatically using one of the supported languages. Additionally once a automaton is created programmatically it can be exported using AP_ExportAnml().

The following code snippets create an automaton that matches the string, "abcd" using a macro definition.

1 <anml version="1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
2  <automata-network id="an1" name="basic ANML automata network">
3  <state-transition-element id="ste1" symbol-set="a" start="start-of-data">
4  <activate-on-match element="u1:p0"/>
5  </state-transition-element>
6  <state-transition-element id="ste2" symbol-set="d">
7  <report-on-match/>
8  </state-transition-element>
9  <macro-reference id="u1" use="m1">
10  <activate-out>
11  <activate-from-macro element="ste2" source="p1"/>
12  </activate-out>
13  </macro-reference>
14  </automata-network>
15 
16  <macro-definition id="m1" name="">
17  <header>
18  <interface-declarations>
19  <port id="p0" type="in"/>
20  <port id="p1" type="out"/>
21  </interface-declarations>
22  </header>
23  <body>
24  <port-definitions>
25  <port-in id="p0">
26  <activate-on-event element="ste1"/>
27  </port-in>
28  <port-out id="p1">
29  <activate-from-event element="ste2"/>
30  </port-out>
31  </port-definitions>
32  <state-transition-element id="ste1" symbol-set="b">
33  <activate-on-match element="ste2"/>
34  </state-transition-element>
35  <state-transition-element id="ste2" symbol-set="c"/>
36  </body>
37  </macro-definition>
38 </anml>
#include <micron/ap/ap_anml.h>
ap_automaton_t CreateAutomaton()
{
ap_anml_t anml;
ap_macro_def_t macroDef;
struct ap_anml_element element;
ap_anml_element_ref_t element_a, element_b, element_c, element_d;
ap_port_ref_t p0, p1;
// initialize the element structure
memset(&element, 0, sizeof(element));
// create the anml object
anml = AP_CreateAnml();
// create the macro definition namespace in the ANML object
AP_CreateMacroDef(anml, &macroDef, 0);
// create the elements that will match "bc" in the macro definition namespace
element.res_type = RT_STE;
element.start = NO_START;
element.symbols = "b";
element.match = 0;
AP_AddAnmlElement(macroDef, &element_b, &element);
element.res_type = RT_STE;
element.start = NO_START;
element.symbols = "c";
element.match = 0;
AP_AddAnmlElement(macroDef, &element_c, &element);
AP_AddAnmlEdge(macroDef, element_b, element_c, 0);
// add an input and output port to the macro definition namespace
AP_AddMacroPort(macroDef, &p0, "p0", PORT_IN);
AP_AddMacroPort(macroDef, &p1, "p1", PORT_OUT);
// attach the input port to the STE that matches "b" and the STE that
// matches "c" to the output port
AP_AttachElementToPort(macroDef, p0, element_b, 0);
AP_AttachElementToPort(macroDef, p1, element_c, 0);
// create the automata network namespace in the ANML object
AP_CreateAutomataNetwork(anml, &anmlNet, 0);
// add STEs that match "a" at the start of data and "d" that reports a match
// to the automata network namespace
element.res_type = RT_STE;
element.start = START_OF_DATA;
element.symbols = "a";
element.match = 0;
AP_AddAnmlElement(anmlNet, &element_a, &element);
element.res_type = RT_STE;
element.start = NO_START;
element.symbols = "d";
element.match = 1;
AP_AddAnmlElement(anmlNet, &element_d, &element);
// create a macro ref that uses the macro definition in the automata network namespace
element.res_type = RT_MACRO_REF;
element.macro_ref = macroDef;
AP_AddAnmlElement(anmlNet, &macroRef, &element);
// connect the STE that matches "a" to the input port of the macro reference
// in the automata network namespace
AP_AddAnmlEdge(anmlNet, element_a, macroRef, p0);
// connect the output port of the macro reference to the STE that matches "d"
// in the automata network namespace
AP_AddAnmlEdgeEx(anmlNet, macroRef, p1, element_d, 0);
// compile the anml object to create the automaton
AP_CompileAnml(anml, &a, 0, 0, 0, 0, 0);
return a;
}
1 from micronap.sdk import *
2 
3 def CreateAutomaton():
4 # create the anml object
5  anml = Anml()
6 
7  # create the macro definition namespace in the ANML object
8  macroDef = anml.CreateMacroDef()
9 
10  # create the elements that will match "bc" in the macro definition namespace
11  element_b = macroDef.AddSTE('b')
12  element_c = macroDef.AddSTE('c')
13  macroDef.AddAnmlEdge(element_b, element_c, 0)
14 
15  # add an input and output port to the macro definition namespace
16  p0 = macroDef.AddMacroPort('p0', AnmlDefs.PORT_IN)
17  p1 = macroDef.AddMacroPort('p1', AnmlDefs.PORT_OUT)
18 
19  # attach the input port to the STE that matches "a" and the STE that
20  # matches "c" to the output port
21  macroDef.AttachElementToPort(p0, element_b, 0)
22  macroDef.AttachElementToPort(p1, element_c, 0)
23 
24  # create the automata network namespace in the ANML object
25  anmlNet = anml.CreateAutomataNetwork()
26 
27  # add STEs that match "a" at the start of data and "d" that reports a match
28  # to the automata network namespace
29  element_a = anmlNet.AddSTE('a', startType=AnmlDefs.START_OF_DATA)
30  element_d = anmlNet.AddSTE('d', match=True)
31 
32  # create a macro ref that uses the macro definition in the automata network namespace
33  macroRef = anmlNet.AddMacroRef(macroDef)
34 
35  # connect the STE that matches "a" to the input port of the macro reference
36  # in the automata network namespace
37  anmlNet.AddAnmlEdge(element_a, macroRef, p0)
38 
39  # connect the output port of the macro reference to the STE that matches "d"
40  # in the automata network namespace
41  anmlNet.AddAnmlEdgeEx(macroRef, p1, element_d, 0)
42 
43  # compile the anml object to create the automaton
44  A,_ = anml.CompileAnml()
45  return A
import com.micron.ap.*;
public class CreateAutomatonExample {
public static Automaton CreateAutomaton() throws ApException {
int p0, p1;
ElementRef element_a, element_b, element_c, element_d, macroRef;
AnmlNetwork anmlNet;
MacroDef macroDef;
// create the anml object
Anml anml = new Anml();
// create the macro definition namespace in the ANML object
macroDef = anml.createMacroDef();
// create the elements that will match "bc" in the macro definition namespace
element_b = macroDef.addSTE("b");
element_c = macroDef.addSTE("c");
macroDef.addAnmlEdge(element_b, element_c, 0);
// add an input and output port to the macro definition namespace
p0 = macroDef.addMacroPort("p0", AnmlDefs.PORT_IN);
p1 = macroDef.addMacroPort("p1", AnmlDefs.PORT_OUT);
// attach the input port to the STE that matches "a" and the STE that
// matches "c" to the output port
macroDef.attachElementToPort(p0, element_b, 0);
macroDef.attachElementToPort(p1, element_c, 0);
// create the automata network namespace in the ANML object
anmlNet = anml.createAutomataNetwork();
// add STEs that match "a" at the start of data and "d" that reports a match
// to the automata network namespace
element_a = anmlNet.addSTE("a", AnmlDefs.START_OF_DATA);
element_d = anmlNet.addSTE("d", AnmlDefs.NO_START, "s1", 1, true);
// create a macro ref that uses the macro definition in the automata network namespace
macroRef = anmlNet.addMacroRef(macroDef);
// connect the STE that matches "a" to the input port of the macro reference
// in the automata network namespace
anmlNet.addAnmlEdge(element_a, macroRef, p0);
// connect the output port of the macro reference to the STE that matches "d"
// in the automata network namespace
anmlNet.addAnmlEdgeEx(macroRef, p1, element_d, 0);
// compile the anml object to create the automaton
return anml.compileAnml(0).getFirst();
}
}

If authoring the ANML XML file, the following code shows how to compile the ANML file.

#include <micron/ap/ap_anml.h>
ap_automaton_t CreateAutomaton(ap_element_map_t* elementMap, const char *filename)
{
ap_anml_network_t anmlNetwork;
ap_anml_t anml;
// create an ANML object in which to build an automata network
anml = AP_CreateAnml();
// Parse the ANML file into the ANML object
AP_LoadAnmlObjects(anml, 0, filename, 0, AP_OPT_DEFAULT);
// compile the automata network into an automaton
AP_CompileAnml(anml, &a, elementMap, 0, 0, 0, 0);
// destroy the ANML object
return a;
}
1 from micronap.sdk import *
2 
3 def CreateAutomaton(filename):
4  # create an ANML obect in which to build an automata network
5  anml = Anml()
6 
7  # Parse the ANML file into the ANML object
8  anml.LoadAnmlObjects(filename)
9 
10  # compile the automata network into an automaton
11  return anml.CompileAnml()
import com.micron.ap.*;
public class CompileAutomatonExample {
public static Automaton CompileAutomaton(String filename) throws ApException {
// create an ANML obect in which to build an automata network
Anml anml = new Anml();
// Parse the ANML file into the ANML object
anml.loadAnmlObjects(filename);
// compile the automata network into an automaton
return anml.compileAnml(0).getFirst();
}
}
1 $ apcompile -A -f output.fsm filename.anml

ANML IDs, References, and the Element Map

Each element within an ANML file must have an ASCII string "ID." It is these IDs that are used to make connections in an ANML file. Although it is more convenient for humans to recognize strings, it is very cumbersome and inefficient for the compiler to internally use strings as identifiers. For this reason, the compiler assigns each ANML element a numeric "reference." When ANML elements are added using AP_AddAnmlElement(), the assigned element reference is returned, and it is this value that will be used to connect with other elements using AP_AddAnmlEdge(). When an ANML object is compiled with AP_CompileAnml() or AP_CompileAnmlFiles(), an entire collection of element references is returned by the compiler in the form of an Element Map. Element maps can be saved to a file by using AP_SaveElementMap() and can be read back by using AP_RestoreElementMap(). The resources used to store an element map can be freed up by using AP_DestroyElementMap(), the size of an element map is returned by AP_GetElementMapSize(), and the attributes of an element map can be retrieved by using AP_GetElementMapInfo(). Any element reference can be converted to an ID by using AP_GetElementIdFromElementRef(), and any element ID can be converted to a reference by using AP_GetElementRefFromElementId(). Any report port reference can be converted to an ID by using AP_GetReportPortIdFromPortRef(), and any report port ID can be converted to a reference by using AP_GetReportPortRefFromPortId(). Element maps are stored in a binary format, so they cannot be viewed directly. In order to view an element map as a text file, us AP_PrintElementMap(). The following is the printed contents of the element map of the automaton created above:

an1.ste1 1
an1.ste2 2
an1.u1 3

The left-hand column contains element IDs and the right-hand column contains element reference values. The automata network's ID is "an1," "ste1" in the automata network namespace has the reference value 1, "ste2" in the automata network namespace has the reference value 2, and the macro definition, "u1," has the reference value 3.