INtime SDK Help
Global objects
The INtime Global Objects feature provides the ability for real-time processes to interact even when resident on different instances of the real-time kernel. The functionality is broadly similar to that provided to Windows applications by the NTX protocol, with enhancements to take into consideration issues of thread priority and multiple client access.
Global Objects technology allows a real-time process to access objects which exist on another instance of the INtime kernel. Each of these "global" objects has a corresponding local object called a reference which points to it and may be used in standard APIs by the local process. For an example, a process can obtain a reference to a mailbox on a different INtime node, INtime then access it using the standard SendRtData and ReceiveRtData APIs. APIs are defined for discovering other nodes, and for creating new global objects, and creating and deleting reference objects.
The Distributed Systems Manager (DSM) organization reflects this system architecture. Prior to INtime version 5, DSM depended on a Windows node to maintain the internal database of relationships. Starting in version 5, this is distributed amongst the participating nodes. A sponsor is identified by its name and also its location and to register a dependency on that sponsor requires knowledge of its name and location.
An INtime system is considered to be a set of one or more nodes. Multiple nodes are connected either through shared memory (multiple processor cores on the same host) or via Ethernet. The Address of a node is described in terms of its Local Logical APIC ID and its network address. The Location of a node is a parameter used by processes on one node to identify another node.
In the case where a multi-core host is connected to the INtime network from one node only, the requests will be forwarded to the target node by the connected node.
Access from Windows to any INtime node is via the NTX API. The underlying network protocol when making NTX requests over the network is shared with the INtime to INtime protocol.
On each node a management process "Global Objects Manager" handles requests on behalf of remote nodes, and manages local connection objects. It is also responsible for managing objects created by other nodes, and the DSM relationships between nodes.
Two objects and an object classification are defined as part of this architecture. The Location object encapsulates information about the location of a given node and the space it occupies. The Reference object contains information allowing the user of its handle to use an object on a remote node using the standard APIs for an object of that type.
The Location object contains APIC ID and network address information for a given node. The handle for a location object (of type LOCATION) has Node scope, not universal scope.
Statically-configured locations are created at kernel boot time and are made visible to other nodes when they boot the first time. These would normally include all other nodes on the local host. Locations on other hosts may be created dynamically or on-demand.
A reference object is created on a node when a process requests access to an object on another node. A reference object can be created explicitly, by a call to CreateRtReferenceObject. Sometimes a reference object is created implicitly, for example by CreateGlobalRtMailbox, or by LookupRtHandle. A reference object may also be deleted explicitly by DeleteRtReferenceObject or implicitly for example by DeleteRtMailbox.
A handle for a reference object has Node scope. The reference object thus represents the nexus between an RTHANDLE for a global object and the LOCATION of that global object. This is particularly important to remember when passing handles between nodes using the extended APIs.
A special instance of a semaphore, mailbox or memory object can be created as a global object, which is a class of object which can be created either on the local node or a remote node and optionally it can be created so that it is not owned by the creating process. The APIs CreateGlobalRtSemaphore, CreateGlobalRtMailbox, CreateGlobalRtMemoryObject and CreateGlobalRtMemoryHandle may be used to create such objects. In the standard case the object created with these APIs is considered to be owned by the creating process, and thus is deleted implicitly when the process is deleted. However such objects may also be created using the REFCOUNT flag, which means that the object is not considered to be owned by the creating process. The object has an associated reference counter which is incremented each time a reference object is created for it and decremented each time one of its reference objects is deleted. The object itself is not deleted until its reference count decrements to zero.
Note that a reference object is always owned by the creating process and will be deleted when that process is deleted.
Hosts are normally automatically identified to other hosts through a HELLO message broadcast on the network. Hosts may also be statically configured by installing a text file named gnethost.cfg containing a list of defined hosts. On a Windows host this is installed in the %INTIMECFG% folder. On a Distributed RTOS host this is installed in the /config folder.
The configuration file may also contain an instruction to change the behavior of HELLO messages. This command, the exclusive command, does the following:
The format of the file is as follows:
# This is a comment line. Anything after a # character is ignored
exclusive=1 # this disables the broadcast of HELLO messages except for hosts listed in this file.
# Any non-zero value enables the exclusive mode.
# All other lines have this format:
# IP-address hostname nodename [nodename]...
# To identify a Windows host, use the special hostname "windows". For example:
192.168.55.14 testhost NodeA NodeB # host "testhost" has NodeA and NodeB
192.168.55.27 windows # this is a Windows host
This lists common operations that allow a real-time process to access objects which exist on another instance of the INtime kernel.
| To . . . | Use this system call . . . |
|---|---|
| Return the first or next LOCATION handle from the local location table | GetFirstRtLocation, GetNextRtLocation |
| Return a LOCATION for the given name | GetRtNodeLocationByName |
| Return a status value indicating the operational state of the target node | GetRtNodeStatus |
| Return information about a node given its location | GetRtNodeInfo |
| Return a handle for a reference to the target node's root process | GetRemoteRootRtProcess |
| Create a reference object from a LOCATION and an RTHANDLE for a valid object on the node indicated by the LOCATION value and return a RTHANDLE for the reference object | CreateRtReferenceObject |
| Delete a reference object | DeleteRtReferenceObject |
| Return information about a reference object and its referenced object | GetRtObjectInfo |
| Create a semaphore on another node indicated by the LOCATION parameter | CreateGlobalRtSemaphore |
| Create a mailbox on another node indicated by the LOCATION parameter | CreateGlobalRtMailbox |
| Create a memory area on another node indicated by the LOCATION parameter and return a handle to a reference object for that area | CreateGlobalRtMemoryObject |
| Create a handle for a physical memory area on another node indicated by the LOCATION parameter, and on the same host as the caller, and return a handle to a reference object for that handle | CreateGlobalRtMemoryHandle |
| Return information to GetRtNodeInfo | NODE_INFO |
| Return information to GetRtObjectInfo | OBJECT_INFO |
| Starts the RT kernel on the specified Node | ntxStartLocalRtNode |
| Stops the RT kernel on the specified Node | ntxStopLocalRtNode |
The following examples illustrate the usage of some of the global objects APIs in order to solve certain problems:
There is an extra sample application available, which is installed in [Documents]\INtime\Projects\globalobjects together with the SDK.
A special location handle THIS_LOCATION (defined in rtdef.h) is used to obtain information about the node which the application is running on.
NODE_INFO info;
// find out about me
if (!GetRtNodeInfo(THIS_LOCATION, &info)) {
printf("Failed to get node info for this location: %04x\n",
GetLastRtError());
return;
}
The NODE_INFO structure returns information about the local node, including the actual location value (instead of the THISLOCATION alias), and the name of the local node.
Find a location value for a given node name and check the status of a node. Note that the BAD_LOCATION value is used to indicate an error from the Location functions.
Finally, get the root process handle for the node, and dump some information about it. Since this creates a reference object it must be deleted when it is no longer needed.
DWORD loc;
RTHANDLE rroot;
OBJECT_INFO oinfo;
loc = GetRtNodeLocationByName(node);
if (loc == BAD_LOCATION) {
printf("Failed to find location %s: %04x\n",
node, (excep = GetLastRtError()));
return excep;
}
if (GetRtNodeStatus(loc) != E_OK) {
printf("Node %s is not running: status %04x\n",
(excep = GetLastRtError()));
return excep;
}
else
printf("Node %s is running\n");
rroot = GetRemoteRootRtProcess(loc);
if (rroot == BAD_RTHANDLE) {
printf("Failed to get remote root process handle: %04x\n",
(excep = GetLastRtError()));
return excep;
}
if (!GetRtObjectInfo(rroot, &oinfo)) {
printf("Failed to get remote root process info: %04x\n",
(excep = GetLastRtError()));
return excep;
}
// Must delete the reference when finished
DeleteRtReferenceObject(rroot);
Get all known locations and list the name of each one.
// find all nodes
printf("List all nodes:\n");
loc = GetFirstRtLocation();
while (loc != BAD_LOCATION) {
if (GetRtNodeInfo(loc, &info))
printf("Loc %d: [%s] node id=%u (%s host)\n",
info.Location, info.szNodeName, info.NodeId,
info.NodeClass == NODE_CLASS_WINDOWS_HOST ?
"Windows" : "INtime");
else
printf("*** Failed to get node info for location %d\n", loc);
loc = GetNextRtLocation(loc);
};
Create a memory object of 1 Mbyte for global access. In this example, we create a global memory object on the local node, for acsess from other processes.
gmem = CreateGlobalRtMemoryObject(THIS_LOCATION, 0x100000, REF_COUNT); // map the memory object p = MapRtSharedMemory(gmem); : // unmap the object before deleting FreeRtMemory(p); // delete the object -- will only delete the memory object // if no one else has it referenced DeleteRtReferenceObject(gmem);
In this example we look up a remote mailbox and send it a handle, expecting a response.
loc = GetRtNodeLocationByName("MachSync");
if (loc == BAD_LOCATION)
printf("Did not find location\n");
return;
}
if (GetRtNodeStatus(loc) == E_OK) {
rRoot = GetRemoteRootRtProcess(loc);
rMbox = LookupRtHandle(rRoot, "ServerMbox", 0);
if (rMbox != BAD_RTHANDLE) {
SendRtHandle(rMbox, hMem, hResp);
if ((h = ReceiveRtHandle(hResp, WAIT_FOREVER, NULL)) == BAD_RTHANDLE)
printf("Failed to receive handle from remote mailbox\n");
else
printf("Received handle %04hx from remote mailbox\n", h);
}
}
}
In this example we obtain a global memory handle and map it to the local process.
rRoot = GetRemoteRootRtProcess(loc);
rMem = LookupRtHandle(rRoot, "DataMemory", 0);
if (rMem != BAD_RTHANDLE) {
p = MapRtSharedMemory(rMem);
// check the buffer (for illustration)
size = GetRtSize(rMem);
if (!ValidateRtBuffer(p, size, BUFFER_WRITABLE))
fprintf(stderr, "Mapped buffer not writable!\n");
else {
// use the memory at p
:
}
// free the mapping when done, and delete the reference
FreeRtMemory(p);
DeleteRtReferenceObject(rMem);
}
This longer sample is designed to run on multiple cores. If the process finds itself on a core with NodeId of 1 then is creates and catalogs a memory object and a semaphore. If it finds itself on any other node it looks up the memory handle and maps it so it can access that memory.
Note that in this case we catalog the actual object handle and not the reference handle, so it can be looked up from other nodes. We could also have created these objects as regular objects and not global objects, but we wanted the REFCOUNT feature.
// The location with NodeId=1 creates the memory and catalogs
// it, the others look it up
if (info.NodeId == 1) {
root = GetRtThreadHandles(ROOT_PROCESS);
// create the memory object with the REFCOUNT flag set
hMem = CreateGlobalRtMemoryObject(THIS_LOCATION, 0x1000, REFCOUNT);
if (hMem == BAD_RTHANDLE) {
fprintf(stderr, "Failed to create global memory object: %04x\n",
GetLastRtError());
return;
}
// Find the actual memory handle
if (!GetRtObjectInfo(hMem, &oinfo)) {
fprintf(stderr, "Could not get info for memory object: %04x\n",
GetLastRtError());
DeleteRtReferenceObject(hMem);
return;
}
// NOTE: we catalog the memory handle, not the reference handle. This
// makes it usable to processes on other nodes.
CatalogRtHandle(root, oinfo.hRemoteObject, MEM_NAME);
// create a semaphore for synchronization and catalog it
hSem = CreateGlobalRtSemaphore(THIS_LOCATION, 0, 1, REFCOUNT);
if (hSem == BAD_RTHANDLE) {
fprintf(stderr, "Could not create global semaphore: %04x\n",
GetLastRtError());
return;
}
// Find the actual semaphore handle
if (!GetRtObjectInfo(hSem, &oinfo)) {
fprintf(stderr, "Could not get info for semaphore: %04x\n",
GetLastRtError());
DeleteRtReferenceObject(hSem);
return;
}
CatalogRtHandle(root, oinfo.hRemoteObject, SEM_NAME);
}
else {
// find the location of the node with NodeId=1
loc = GetFirstRtLocation();
if (loc == BAD_LOCATION) {
fprintf(stderr, "Could not get first location: %04x\n",
GetLastRtError());
return;
}
do {
if (!GetRtNodeInfo(loc, &info)) {
fprintf(stderr, "Could not get info for loc %d: %04x\n",
loc, GetLastRtError());
return;
}
if (info.NodeId == 1)
break;
loc = GetNextRtLocation(loc);
} while (loc != BAD_LOCATION);
if (loc == BAD_LOCATION) {
fprintf(stderr, "Could not find node with ID 1\n");
return;
}
// Get the root process handle and look up the memory object
root = GetRemoteRootRtProcess(loc);
printf("Waiting for objects to be cataloged\n");
hMem = LookupRtHandle(root, MEM_NAME, WAIT_FOREVER);
if (hMem == BAD_RTHANDLE) {
fprintf(stderr, "Failed to lookup memory handle: %04x\n",
GetLastRtError()); return;
}
hSem = LookupRtHandle(root, SEM_NAME, WAIT_FOREVER);
if (hSem == BAD_RTHANDLE) {
fprintf(stderr, "Failed to lookup semaphore handle: %04x\n",
GetLastRtError()); return;
}
}
// Now map the memory
p = MapRtSharedMemory(hMem); if (p == NULL) {
fprintf(stderr, "Could not map memory object %04x: %04x\n",
hMem, GetLastRtError());
type = GetRtHandleTypeEx(hMem, &subtype);
fprintf(stderr, "hMemtype = %u, subtype %04x\n", type, subtype);
return;
}